Jul 10 14:06:33.505626 ip-10-0-143-123 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 10 14:06:33.505638 ip-10-0-143-123 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 10 14:06:33.505645 ip-10-0-143-123 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 10 14:06:33.505894 ip-10-0-143-123 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 10 14:06:43.531579 ip-10-0-143-123 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 10 14:06:43.531595 ip-10-0-143-123 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d5a19f8b3740473f915fa7b5b921af02 --
Jul 10 14:08:48.254046 ip-10-0-143-123 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 14:08:48.645902 ip-10-0-143-123 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 14:08:48.645902 ip-10-0-143-123 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 14:08:48.645902 ip-10-0-143-123 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 14:08:48.645902 ip-10-0-143-123 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 14:08:48.645902 ip-10-0-143-123 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 14:08:48.646994 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.646902    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 14:08:48.652993 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.652973    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:08:48.652993 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.652990    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:08:48.652993 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.652997    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:08:48.652993 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653003    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653008    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653013    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653017    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653021    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653025    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653029    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653033    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653037    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653052    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653057    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653062    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653066    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653071    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653075    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653079    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653083    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653087    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653092    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:08:48.653244 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653096    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653103    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653109    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653114    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653120    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653124    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653129    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653134    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653139    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653144    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653149    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653154    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653158    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653164    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653168    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653173    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653178    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653183    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653187    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653193    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:08:48.654032 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653198    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653202    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653207    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653211    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653216    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653221    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653226    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653230    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653235    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653239    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653243    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653248    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653252    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653257    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653261    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653266    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653270    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653275    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653282    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653289    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:08:48.654571 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653294    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653302    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653310    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653316    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653321    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653327    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653332    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653339    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653344    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653349    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653354    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653358    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653364    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653369    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653374    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653378    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653383    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653388    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653393    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653398    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:08:48.655368 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653403    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653407    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653412    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653416    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653420    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653425    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653430    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653435    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653439    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653444    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653449    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653453    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653459    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653464    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653468    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653474    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.653479    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655644    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655656    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655661    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:08:48.656264 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655666    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655670    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655675    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655680    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655685    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655690    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655694    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655699    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655704    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655708    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655713    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655717    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655722    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655727    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655732    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655736    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655741    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655746    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655751    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655755    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655760    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:08:48.656827 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655764    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655769    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655773    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655778    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655784    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655812    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655819    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655825    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655830    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655835    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655839    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655846    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655851    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655857    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655863    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655868    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655872    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655877    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655882    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:08:48.657748 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655886    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655890    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655895    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655899    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655903    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655909    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655914    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655919    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655923    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655927    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655932    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655937    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655941    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655945    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655950    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655954    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655960    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655965    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655970    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:08:48.658687 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655975    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655982    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655989    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655995    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.655999    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656004    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656008    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656013    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656017    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656021    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656025    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656030    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656034    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656039    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656044    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656048    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656053    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656058    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656063    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:08:48.659586 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656068    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656072    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656076    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656081    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656085    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656090    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656097    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656102    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656106    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656111    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656116    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656121    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656125    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656130    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656135    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656140    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656145    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.656150    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657403    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657415    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:08:48.660223 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657422    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657427    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657433    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657438    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657443    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657448    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657453    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657457    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657462    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657466    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657472    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657477    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657482    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657486    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657491    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657495    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657500    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657504    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657510    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657514    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:08:48.660959 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657519    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657524    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657528    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657533    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657542    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657549    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657554    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657559    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657564    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657569    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657573    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657578    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657583    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657587    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657591    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657596    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657601    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657606    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657610    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:08:48.661526 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657615    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657619    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657624    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657628    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657633    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657637    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657641    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657646    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657650    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657655    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657661    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657667    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657673    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657678    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657682    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657687    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657691    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657696    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657701    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657705    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:08:48.662041 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657710    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657714    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657720    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657726    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657731    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657735    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657739    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657744    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657749    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657753    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657757    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657761    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657765    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657769    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657775    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657779    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657784    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657788    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657811    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657816    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:08:48.662657 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657820    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657825    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657833    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657838    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657843    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657848    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657853    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657857    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657862    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657867    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657871    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657876    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657880    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657885    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657889    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657897    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657903    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.657908    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658010    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658022    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 14:08:48.663179 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658033    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658040    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658047    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658052    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658059    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658067    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658073    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658078    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658084    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658090    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658095    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658100    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658105    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658111    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658115    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658121    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658128    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658133    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658139    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658144    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658150    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658157    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658162    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 14:08:48.663694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658168    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658174    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658179    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658184    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658189    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658195    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658200    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658207    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658213    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658219    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658224    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658230    2584 flags.go:64] FLAG: --enable-server="true"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658235    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658242    2584 flags.go:64] FLAG: --event-burst="100"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658248    2584 flags.go:64] FLAG: --event-qps="50"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658253    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658258    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658264    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658281    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658287    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658292    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658297    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658302    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658307    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658312    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 14:08:48.664293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658318    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658322    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658327    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658332    2584 flags.go:64] FLAG: --feature-gates=""
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658339    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658360    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658367    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658373    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658378    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658383    2584 flags.go:64] FLAG: --help="false"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658389    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-143-123.ec2.internal"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658394    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658399    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658412    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658419    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658428    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658433    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658438    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658445    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658450    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658455    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658462    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658467    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658472    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 14:08:48.664978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658476    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658482    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658487    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658492    2584 flags.go:64] FLAG: --lock-file=""
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658497    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658502    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658507    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658517    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658521    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658527    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658532    2584 flags.go:64] FLAG: --logging-format="text"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658537    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658543    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658547    2584 flags.go:64] FLAG: --manifest-url=""
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658552    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658560    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658565    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658572    2584 flags.go:64] FLAG: --max-pods="110"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658577    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658582    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658587    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658592    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658608    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658616    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658622    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 14:08:48.665624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658633    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658641    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658646    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658651    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658656    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658666    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658671    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658676    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658681    2584 flags.go:64] FLAG: --port="10250"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658686    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658692    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d1094d98477c8bca"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658697    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658702    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658707    2584 flags.go:64] FLAG: --register-node="true"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658713    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658719    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658724    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658730    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658735    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658741    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658747    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658752    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658757    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658762    2584 flags.go:64] FLAG: --runonce="false"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658766    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 14:08:48.666285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658772    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658778    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658783    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658788    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658810    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658826    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658833    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658837    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658843    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658850    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658854    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658860    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658865    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658871    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658880    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658885    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658890    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658902    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658907    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658912    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658917    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658922    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658927    2584 flags.go:64] FLAG: --v="2"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658935    2584 flags.go:64] FLAG: --version="false"
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658942    2584 flags.go:64] FLAG: --vmodule=""
Jul 10 14:08:48.667041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658948    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.658953    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659168    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659175    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659180    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659185    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659191    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659196    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659200    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659205    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659210    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659214    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659222    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659237    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659243    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659247    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659253    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659258    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659262    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659267    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:08:48.668044 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659272    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659276    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659283    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659289    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659294    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659299    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659304    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659309    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659314    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659318    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659323    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659327    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659332    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659336    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659341    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659346    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659350    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659355    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659359    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:08:48.668568 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659363    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659368    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659372    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659377    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659381    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659387    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659392    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659405    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659410    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659416    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659424    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659429    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659434    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659439    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659444    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659448    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659452    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659457    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659461    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659466    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:08:48.669180 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659470    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659475    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659479    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659484    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659488    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659493    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659497    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659501    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659506    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659510    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659515    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659520    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659524    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659528    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659533    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659537    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659542    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659548    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659553    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659558    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:08:48.669696 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659572    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659579    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659583    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659588    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659592    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659596    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659601    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659605    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659610    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659614    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659619    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659623    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659628    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659633    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659637    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659641    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659646    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659651    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659655    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659660    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:08:48.670241 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659678    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.659683    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.659691    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.668336    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.668353    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.668380    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.668390    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668451    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668457    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668461    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668464    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668468    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668471    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668474    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668477    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:08:48.670755 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668480    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668483    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668486    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668489    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668493    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668495    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668498    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668501    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668503    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668506    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668509    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668513    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668516    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668518    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668521    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668524    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668527    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668530    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668533    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668537    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:08:48.671175 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668539    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668543    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668546    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668549    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668552    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668562    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668565    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668567    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668570    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668573    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668576    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668579    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668582    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668585    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668588    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668591    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668593    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668596    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668599    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668602    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:08:48.671708 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668605    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668608    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668610    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668613    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668616    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668618    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668621    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668624    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668626    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668629    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668632    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668634    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668637    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668640    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668643    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668646    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668649    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668652    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668654    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668657    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:08:48.672297 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668660    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668663    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668665    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668668    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668671    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668673    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668676    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668679    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668682    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668684    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668689    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668694    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668697    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668700    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668702    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668705    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668708    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668711    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668715    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:08:48.672814 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668719    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668722    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668725    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668727    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668730    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668733    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668736    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668740    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668744    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668747    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668750    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668753    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.668760    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668922    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 14:08:48.673304 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668930    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668934    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668937    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668940    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668942    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668945    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668948    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668951    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668954    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668957    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668959    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668963    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668966    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668969    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668971    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668974    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668976    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668979    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668982    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668984    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 14:08:48.673718 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668987    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668989    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668992    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668995    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.668998    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669001    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669004    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669007    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669009    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669013    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669017    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669020    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669023    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669025    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669028    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669031    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669033    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669036    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669038    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669041    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 14:08:48.674242 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669043    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669047    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669049    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669052    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669056    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669058    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669061    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669064    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669066    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669069    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669072    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669075    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669077    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669080    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669083    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669086    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669088    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669091    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669094    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 14:08:48.674752 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669098    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669101    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669104    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669109    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669112    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669115    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669118    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669121    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669123    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669126    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669129    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669131    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669134    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669137    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669140    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669142    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669145    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669148    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669152    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 14:08:48.675271 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669154    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669157    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669160    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669163    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669166    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669168    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669171    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669174    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669177    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669179    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669182    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669185    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669188    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669191    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669194    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669197    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669200    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669203    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669205    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 14:08:48.675789 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:48.669208    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 14:08:48.676322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.669213    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 14:08:48.676322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.669337    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 14:08:48.676322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.671638    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 14:08:48.676322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.673121    2584 server.go:1019] "Starting client certificate rotation"
Jul 10 14:08:48.676322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.673233    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 14:08:48.676322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.673266    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 14:08:48.676322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.675662    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 14:08:48.677077 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.677059    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 14:08:48.686664 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.686643    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 10 14:08:48.690490 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.690473    2584 log.go:25] "Validated CRI v1 image API"
Jul 10 14:08:48.692218 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.692191    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 14:08:48.696317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.696298    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c98eb120-3331-41f3-81b8-592285376314:/dev/nvme0n1p3 d128ddf4-39f0-473d-8cb0-c4e6f7a41dbb:/dev/nvme0n1p4]
Jul 10 14:08:48.696396 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.696316    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 14:08:48.702007 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.701884    2584 manager.go:216] Machine: {Timestamp:2026-07-10 14:08:48.700555065 +0000 UTC m=+0.326280990 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092994 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26c3983f6b380d9d90482d40169995 SystemUUID:ec26c398-3f6b-380d-9d90-482d40169995 BootID:d5a19f8b-3740-473f-915f-a7b5b921af02 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:35:91:f5:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:35:91:f5:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:63:8a:23:79:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 14:08:48.702007 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.701996    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 14:08:48.702130 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.702110    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 14:08:48.704532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.704513    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 14:08:48.704993 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.704962    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 14:08:48.705135 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.704997    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 14:08:48.706317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.706306    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 14:08:48.706359 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.706319    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 14:08:48.706359 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.706327    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 14:08:48.706974 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.706963    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 14:08:48.708430 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.708420    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 14:08:48.711806 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.711783    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 14:08:48.711926 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.711916    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 14:08:48.711959 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.711946    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 14:08:48.711959 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.711955    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 14:08:48.712012 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.711974    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 14:08:48.712012 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.711986    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 10 14:08:48.712012 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.711994    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 14:08:48.712429 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.712412    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 14:08:48.712503 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.712448    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 14:08:48.716712 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.716691    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 14:08:48.718737 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718721    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 14:08:48.718818 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718754    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 14:08:48.718949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718925    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 14:08:48.718984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718952    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 14:08:48.718984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718959    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 14:08:48.718984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718965    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 14:08:48.718984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718974    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 14:08:48.718984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.718980    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 14:08:48.719501 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.719492    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 14:08:48.719501 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.719501    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 14:08:48.719574 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.719508    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 14:08:48.719574 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.719514    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 14:08:48.719574 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.719522    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 14:08:48.719574 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.719531    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 14:08:48.720239 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.720229    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 14:08:48.720278 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.720239    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 14:08:48.723610 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.723595    2584 server.go:1268] "Started kubelet"
Jul 10 14:08:48.723768 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.723740    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 14:08:48.723825 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.723770    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 14:08:48.723872 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.723842    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 14:08:48.724947 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.724926    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 14:08:48.725017 ip-10-0-143-123 systemd[1]: Started Kubernetes Kubelet.
Jul 10 14:08:48.726274 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.726255    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 14:08:48.727835 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.727807    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 14:08:48.728570 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.728552    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 14:08:48.728651 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.728576    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 14:08:48.728651 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.728617    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 14:08:48.729698 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.728556    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-123.ec2.internal.18c0f2e78b162a6c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-123.ec2.internal,UID:ip-10-0-143-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-123.ec2.internal,},FirstTimestamp:2026-07-10 14:08:48.72356926 +0000 UTC m=+0.349295185,LastTimestamp:2026-07-10 14:08:48.72356926 +0000 UTC m=+0.349295185,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-123.ec2.internal,}"
Jul 10 14:08:48.732208 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.732191    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-65vvf"
Jul 10 14:08:48.732504 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.732488    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 14:08:48.732560 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.732503    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 14:08:48.732611 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.732563    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:48.733294 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733266    2584 factory.go:55] Registering systemd factory
Jul 10 14:08:48.733388 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733298    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 10 14:08:48.733522 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733504    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 14:08:48.733596 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733508    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 14:08:48.733596 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733537    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 14:08:48.733596 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733569    2584 factory.go:153] Registering CRI-O factory
Jul 10 14:08:48.733596 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733580    2584 factory.go:223] Registration of the crio container factory successfully
Jul 10 14:08:48.733795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733622    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 14:08:48.733795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733643    2584 factory.go:103] Registering Raw factory
Jul 10 14:08:48.733795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733654    2584 manager.go:1196] Started watching for new ooms in manager
Jul 10 14:08:48.733795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733661    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 14:08:48.733795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.733672    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 14:08:48.734092 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.734004    2584 manager.go:349] Starting recovery of all containers
Jul 10 14:08:48.734772 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.734666    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 14:08:48.734772 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.734723    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 14:08:48.740909 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.740636    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-65vvf"
Jul 10 14:08:48.743559 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.743530    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 14:08:48.747888 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.747867    2584 manager.go:354] Recovery completed
Jul 10 14:08:48.752778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.752764    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:08:48.755304 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.755286    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:08:48.755372 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.755318    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:08:48.755372 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.755329    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:08:48.755948 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.755932    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 14:08:48.755948 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.755946    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 14:08:48.755948 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.755949    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:08:48.756153 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.755965    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 14:08:48.756584 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.756570    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:08:48.756632 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.756598    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:08:48.756632 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.756613    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:08:48.757242 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.757228    2584 policy_none.go:49] "None policy: Start"
Jul 10 14:08:48.757305 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.757248    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 14:08:48.757305 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.757262    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 14:08:48.758505 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.758490    2584 policy_none.go:47] "Start"
Jul 10 14:08:48.793386 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.793362    2584 manager.go:337] "Starting Device Plugin manager"
Jul 10 14:08:48.793498 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.793395    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 14:08:48.793498 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.793406    2584 server.go:87] "Starting device plugin registration server"
Jul 10 14:08:48.793643 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.793632    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 14:08:48.793680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.793645    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 14:08:48.793789 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.793772    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 14:08:48.793878 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.793857    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 14:08:48.793878 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.793867    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 14:08:48.794397 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.794377    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 14:08:48.794474 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.794425    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:48.894372 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.894331    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:08:48.895774 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.895593    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:08:48.895774 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.895627    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:08:48.895774 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.895642    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:08:48.895774 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.895678    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-123.ec2.internal"
Jul 10 14:08:48.903968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.903947    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-123.ec2.internal"
Jul 10 14:08:48.904078 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.903979    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-123.ec2.internal\": node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:48.914219 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.914192    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 14:08:48.914317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.914231    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 14:08:48.914317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.914251    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 14:08:48.914317 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.914284    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 14:08:48.916925 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:48.916908    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:08:48.920383 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:48.920362    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.015672 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.015581    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal"]
Jul 10 14:08:49.015672 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.015680    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:08:49.017173 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.017157    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:08:49.017258 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.017185    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:08:49.017258 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.017197    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:08:49.018479 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.018465    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:08:49.018618 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.018599    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.018661 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.018639    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:08:49.020804 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.020783    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.021202 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.021187    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:08:49.021269 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.021215    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:08:49.021269 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.021225    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:08:49.021342 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.021299    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:08:49.021342 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.021330    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:08:49.021342 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.021343    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:08:49.022535 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.022521    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.022624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.022551    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 14:08:49.023210 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.023191    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 14:08:49.023314 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.023224    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 14:08:49.023314 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.023236    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeHasSufficientPID"
Jul 10 14:08:49.035535 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.035515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b301e1fb865033315caa9abfb1e6ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal\" (UID: \"9b301e1fb865033315caa9abfb1e6ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.035623 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.035539    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b301e1fb865033315caa9abfb1e6ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal\" (UID: \"9b301e1fb865033315caa9abfb1e6ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.035623 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.035557    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7490a82eb930c707bbdadeb01bf134d0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-123.ec2.internal\" (UID: \"7490a82eb930c707bbdadeb01bf134d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.051730 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.051697    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-123.ec2.internal\" not found" node="ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.056627 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.056608    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-123.ec2.internal\" not found" node="ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.121490 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.121461    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.135724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.135698    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b301e1fb865033315caa9abfb1e6ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal\" (UID: \"9b301e1fb865033315caa9abfb1e6ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.135805 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.135744    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b301e1fb865033315caa9abfb1e6ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal\" (UID: \"9b301e1fb865033315caa9abfb1e6ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.135805 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.135771    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7490a82eb930c707bbdadeb01bf134d0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-123.ec2.internal\" (UID: \"7490a82eb930c707bbdadeb01bf134d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.135906 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.135803    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7490a82eb930c707bbdadeb01bf134d0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-123.ec2.internal\" (UID: \"7490a82eb930c707bbdadeb01bf134d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.135906 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.135815    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b301e1fb865033315caa9abfb1e6ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal\" (UID: \"9b301e1fb865033315caa9abfb1e6ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.135906 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.135820    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b301e1fb865033315caa9abfb1e6ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal\" (UID: \"9b301e1fb865033315caa9abfb1e6ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.222273 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.222243    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.328969 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.328889    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.354257 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.354222    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.359487 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.359469    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:49.430529 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.430487    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.538118 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.538088    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.638627 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.638546    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.674399 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.674372    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 14:08:49.674978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.674530    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 14:08:49.723006 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.722979    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:08:49.738936 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.738910    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.745348 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.745322    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 14:03:49 +0000 UTC" deadline="2028-04-15 00:26:16.43983496 +0000 UTC"
Jul 10 14:08:49.745414 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.745348    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15466h17m26.694490356s"
Jul 10 14:08:49.839791 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.839759    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:49.842366 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:49.842349    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:08:49.940689 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:49.940659    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:50.017964 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:50.017920    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b301e1fb865033315caa9abfb1e6ef6.slice/crio-ca93b7adcac1343d7f13e448634e089d215e394956456a051182198dc03efe31 WatchSource:0}: Error finding container ca93b7adcac1343d7f13e448634e089d215e394956456a051182198dc03efe31: Status 404 returned error can't find the container with id ca93b7adcac1343d7f13e448634e089d215e394956456a051182198dc03efe31
Jul 10 14:08:50.018146 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:50.018125    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7490a82eb930c707bbdadeb01bf134d0.slice/crio-0493f069fa541350d4cdaad22ef462d475162f0b5361dac9badf1df7fcbfbf37 WatchSource:0}: Error finding container 0493f069fa541350d4cdaad22ef462d475162f0b5361dac9badf1df7fcbfbf37: Status 404 returned error can't find the container with id 0493f069fa541350d4cdaad22ef462d475162f0b5361dac9badf1df7fcbfbf37
Jul 10 14:08:50.022062 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.022046    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:08:50.041078 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.041059    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:50.141841 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.141803    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:50.242096 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.242013    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-123.ec2.internal\" not found"
Jul 10 14:08:50.263835 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.263809    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:08:50.333666 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.333639    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:50.347628 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.347611    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 14:08:50.348655 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.348642    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal"
Jul 10 14:08:50.359785 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.359770    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 14:08:50.713096 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.713015    2584 apiserver.go:52] "Watching apiserver"
Jul 10 14:08:50.724183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.724150    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 14:08:50.726174 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.726140    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-5j7xb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg","openshift-cluster-node-tuning-operator/tuned-jl5q6","openshift-multus/multus-p4wm7","openshift-network-operator/iptables-alerter-sjvb7","kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal","openshift-dns/node-resolver-t2kz9","openshift-image-registry/node-ca-qjk28","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal","openshift-multus/multus-additional-cni-plugins-q6kjx","openshift-multus/network-metrics-daemon-qmqv2","openshift-network-diagnostics/network-check-target-5trbg","openshift-ovn-kubernetes/ovnkube-node-8hz6t","kube-system/global-pull-secret-syncer-cm2np"]
Jul 10 14:08:50.727831 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.727807    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.729266 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.729233    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.730527 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.730508    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.731099 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.731079    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 14:08:50.731191 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.731168    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9rwzb\""
Jul 10 14:08:50.731706 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.731675    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 14:08:50.731823 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.731787    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.732335 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.732313    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 14:08:50.732428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.732372    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 14:08:50.732493 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.732471    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x6fvb\""
Jul 10 14:08:50.733224 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.733203    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 14:08:50.733316 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.733256    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.733652 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.733633    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 14:08:50.734128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.733862    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 14:08:50.734128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.733975    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 14:08:50.734288 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.734245    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 14:08:50.734345 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.734320    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4rz7w\""
Jul 10 14:08:50.734512 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.734480    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 14:08:50.734595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.734559    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 14:08:50.734901 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.734883    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 14:08:50.735657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.735365    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.735657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.735453    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rgq5t\""
Jul 10 14:08:50.735657 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.735461    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:08:50.735886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.735730    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 14:08:50.735886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.735777    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 14:08:50.737132 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.736958    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 14:08:50.737132 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.737031    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 14:08:50.737132 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.737120    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 14:08:50.737496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.737454    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vcdfm\""
Jul 10 14:08:50.737933 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.737915    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.738799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.738780    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 14:08:50.739605 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.739584    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:50.740829 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.740810    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 14:08:50.740960 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.740941    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 14:08:50.741042 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.741023    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.741098 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.740878    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-44z9s\""
Jul 10 14:08:50.741289 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.741263    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 14:08:50.742734 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.742702    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 14:08:50.742845 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.742807    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 14:08:50.742915 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.742708    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6w5gk\""
Jul 10 14:08:50.743138 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.743117    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.744243 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.744222    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:50.744343 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.744287    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:08:50.744397 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.744375    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 14:08:50.745290 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.744426    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-k8s-cni-cncf-io\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.745397 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.745334    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-conf-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.745567 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.745468    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 14:08:50.745567 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.745531    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.745761    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwtlf\" (UniqueName: \"kubernetes.io/projected/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-kube-api-access-zwtlf\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.745829    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-slash\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746005    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-env-overrides\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746039    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-lib-modules\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746074    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-host\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746103    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746276    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-cni-netd\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746345    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-log-socket\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746405    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-cni-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746438    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-iptables-alerter-script\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746467    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cv5c6\""
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746505    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovn-node-metrics-cert\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746548    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovnkube-script-lib\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746596    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/259a5382-782c-4cba-8bee-effd8d3b49fc-kubelet-config\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-tmp-dir\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.746895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746853    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-modprobe-d\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746916    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-systemd\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746928    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.746964    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-ovn\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747067    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3da23668-9375-4dd5-9332-e7e047d0bd7c-tmp\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747095    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjtp\" (UniqueName: \"kubernetes.io/projected/3da23668-9375-4dd5-9332-e7e047d0bd7c-kube-api-access-dsjtp\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747244    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747280    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-var-lib-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747419    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-48bkj\""
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747628    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747681    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-cnibin\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747720    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-kubelet\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.747778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747769    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74mr4\" (UniqueName: \"kubernetes.io/projected/e7df77de-0042-4de4-8622-715fd060430e-kube-api-access-74mr4\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747822    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/259a5382-782c-4cba-8bee-effd8d3b49fc-dbus\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747880    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-cni-bin\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747923    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysctl-d\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.747967    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-var-lib-kubelet\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748016    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-os-release\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748069    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-etc-kubernetes\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748106    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-run-netns\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748160    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-hosts-file\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748198    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-kubelet\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748229    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-etc-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748280    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-run\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748314    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-system-cni-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748345    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvj5r\" (UniqueName: \"kubernetes.io/projected/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-kube-api-access-qvj5r\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.748398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748373    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovnkube-config\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748437    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-tuned\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748469    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-socket-dir-parent\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7df77de-0042-4de4-8622-715fd060430e-multus-daemon-config\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748543    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-systemd-units\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7df77de-0042-4de4-8622-715fd060430e-cni-binary-copy\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748825    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-netns\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.748912    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.748971    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-cni-multus\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.749127 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749055    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-multus-certs\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749118    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-host-slash\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749197    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-systemd\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749237    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysctl-conf\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749284    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-cni-bin\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749349    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysconfig\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749384    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-kubernetes\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749430    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzjg9\" (UniqueName: \"kubernetes.io/projected/597072ba-b1e2-4d2e-baa8-a253fefb6284-kube-api-access-nzjg9\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749558    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-sys\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.749646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749620    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-hostroot\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.750130 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749675    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-node-log\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.750130 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.749723    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.751384 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.751363    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 14:08:50.793731 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.793706    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tlzcl"
Jul 10 14:08:50.807319 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.807288    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tlzcl"
Jul 10 14:08:50.835343 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.835305    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 14:08:50.850224 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-kubelet\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.850394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850233    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74mr4\" (UniqueName: \"kubernetes.io/projected/e7df77de-0042-4de4-8622-715fd060430e-kube-api-access-74mr4\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.850394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850266    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.850394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850293    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmm46\" (UniqueName: \"kubernetes.io/projected/cf0867b7-2f9c-4963-9a99-2c02c22e6267-kube-api-access-tmm46\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.850394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850321    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/259a5382-782c-4cba-8bee-effd8d3b49fc-dbus\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.850394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-cni-bin\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.850394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysctl-d\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850405    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-var-lib-kubelet\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850418    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-kubelet\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850430    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-os-release\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850455    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-etc-kubernetes\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850482    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-system-cni-dir\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850508    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-run-netns\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850533    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-hosts-file\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850549    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/259a5382-782c-4cba-8bee-effd8d3b49fc-dbus\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850571    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysctl-d\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850558    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-kubelet\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850609    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-kubelet\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850621    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-cni-bin\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.850631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850622    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-var-lib-kubelet\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850681    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-os-release\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850705    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-etc-kubernetes\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850703    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-run-netns\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850722    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-hosts-file\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-etc-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-etc-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850794    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-run\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850819    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-system-cni-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850860    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvj5r\" (UniqueName: \"kubernetes.io/projected/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-kube-api-access-qvj5r\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850872    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-run\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovnkube-config\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850910    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-tuned\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850926    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-socket-dir-parent\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850951    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-system-cni-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-socket-dir-parent\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.850999    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7df77de-0042-4de4-8622-715fd060430e-multus-daemon-config\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851028    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-device-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.851188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851053    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-sys-fs\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851079    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-systemd-units\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851120    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7df77de-0042-4de4-8622-715fd060430e-cni-binary-copy\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851162    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-netns\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851177    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-systemd-units\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851191    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-cni-multus\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851216    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-multus-certs\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851226    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-netns\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851240    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-host-slash\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851267    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfmxm\" (UniqueName: \"kubernetes.io/projected/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-kube-api-access-kfmxm\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851275    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-multus-certs\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851271    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-cni-multus\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851323    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g5z8\" (UniqueName: \"kubernetes.io/projected/fb6959d7-ed34-4fe4-9897-12ea79409901-kube-api-access-7g5z8\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851367    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-systemd\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851383    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-host-slash\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851393    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysctl-conf\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851418    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-cni-bin\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.851863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851439    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-systemd\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851444    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851474    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cae3993c-e09a-4c84-8d16-6eed8dff45dd-agent-certs\") pod \"konnectivity-agent-5j7xb\" (UID: \"cae3993c-e09a-4c84-8d16-6eed8dff45dd\") " pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-os-release\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851525    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysconfig\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851554    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-kubernetes\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851558    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysctl-conf\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-serviceca\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851629    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851655    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-socket-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851648    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-kubernetes\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851686    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7df77de-0042-4de4-8622-715fd060430e-cni-binary-copy\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851654    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovnkube-config\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851703    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e7df77de-0042-4de4-8622-715fd060430e-multus-daemon-config\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzjg9\" (UniqueName: \"kubernetes.io/projected/597072ba-b1e2-4d2e-baa8-a253fefb6284-kube-api-access-nzjg9\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851843    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-sys\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851867    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-hostroot\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.852724 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851856    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-node-log\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851904    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-sysconfig\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851930    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-sys\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851941    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-var-lib-cni-bin\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.851998    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852007    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-hostroot\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852040    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-node-log\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852083    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-k8s-cni-cncf-io\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852112    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-host-run-k8s-cni-cncf-io\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852160    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852189    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-conf-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852220    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwtlf\" (UniqueName: \"kubernetes.io/projected/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-kube-api-access-zwtlf\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852248    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852289    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-slash\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-env-overrides\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852335    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-lib-modules\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.853600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852361    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-host\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852389    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852399    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-conf-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852467    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-etc-selinux\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852504    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852511    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-slash\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852532    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-cni-netd\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852475    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-host\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852634    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852655    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-lib-modules\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852670    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-host\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852696    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-host-cni-netd\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852733    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-registration-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852786    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852824    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-log-socket\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852858    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-cni-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852874    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-log-socket\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.854470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852891    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-env-overrides\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852927    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-iptables-alerter-script\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852959    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.852989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovn-node-metrics-cert\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853037    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovnkube-script-lib\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853040    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-multus-cni-dir\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853071    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/259a5382-782c-4cba-8bee-effd8d3b49fc-kubelet-config\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853100    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-tmp-dir\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853141    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-modprobe-d\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853177    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/259a5382-782c-4cba-8bee-effd8d3b49fc-kubelet-config\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853232    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-systemd\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853260    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-modprobe-d\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853517    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-iptables-alerter-script\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853560    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-tmp-dir\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853183    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-systemd\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853739    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwg7s\" (UniqueName: \"kubernetes.io/projected/2b40128e-bfe7-46ea-baa5-edd38a752eb7-kube-api-access-qwg7s\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853785    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cae3993c-e09a-4c84-8d16-6eed8dff45dd-konnectivity-ca\") pod \"konnectivity-agent-5j7xb\" (UID: \"cae3993c-e09a-4c84-8d16-6eed8dff45dd\") " pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:50.855267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853827    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-ovn\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853860    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3da23668-9375-4dd5-9332-e7e047d0bd7c-tmp\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjtp\" (UniqueName: \"kubernetes.io/projected/3da23668-9375-4dd5-9332-e7e047d0bd7c-kube-api-access-dsjtp\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853892    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovnkube-script-lib\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853925    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-run-ovn\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853950    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.853979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cnibin\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.854009    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.854036    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-var-lib-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.854062    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-cnibin\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.854157    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7df77de-0042-4de4-8622-715fd060430e-cnibin\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.854199    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/597072ba-b1e2-4d2e-baa8-a253fefb6284-var-lib-openvswitch\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.854269    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.854348    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret podName:259a5382-782c-4cba-8bee-effd8d3b49fc nodeName:}" failed. No retries permitted until 2026-07-10 14:08:51.354326442 +0000 UTC m=+2.980052357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret") pod "global-pull-secret-syncer-cm2np" (UID: "259a5382-782c-4cba-8bee-effd8d3b49fc") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:50.855968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.855310    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3da23668-9375-4dd5-9332-e7e047d0bd7c-etc-tuned\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.856594 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.856001    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/597072ba-b1e2-4d2e-baa8-a253fefb6284-ovn-node-metrics-cert\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.856594 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.856569    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3da23668-9375-4dd5-9332-e7e047d0bd7c-tmp\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.860135 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.860108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74mr4\" (UniqueName: \"kubernetes.io/projected/e7df77de-0042-4de4-8622-715fd060430e-kube-api-access-74mr4\") pod \"multus-p4wm7\" (UID: \"e7df77de-0042-4de4-8622-715fd060430e\") " pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:50.860272 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.860253    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvj5r\" (UniqueName: \"kubernetes.io/projected/887b43ac-8a5f-44e7-8f3b-d90ccd70225b-kube-api-access-qvj5r\") pod \"iptables-alerter-sjvb7\" (UID: \"887b43ac-8a5f-44e7-8f3b-d90ccd70225b\") " pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:50.860595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.860580    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzjg9\" (UniqueName: \"kubernetes.io/projected/597072ba-b1e2-4d2e-baa8-a253fefb6284-kube-api-access-nzjg9\") pod \"ovnkube-node-8hz6t\" (UID: \"597072ba-b1e2-4d2e-baa8-a253fefb6284\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:50.861434 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.861419    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwtlf\" (UniqueName: \"kubernetes.io/projected/b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d-kube-api-access-zwtlf\") pod \"node-resolver-t2kz9\" (UID: \"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d\") " pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:50.864850 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.864833    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjtp\" (UniqueName: \"kubernetes.io/projected/3da23668-9375-4dd5-9332-e7e047d0bd7c-kube-api-access-dsjtp\") pod \"tuned-jl5q6\" (UID: \"3da23668-9375-4dd5-9332-e7e047d0bd7c\") " pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:50.920197 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.920144    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal" event={"ID":"7490a82eb930c707bbdadeb01bf134d0","Type":"ContainerStarted","Data":"0493f069fa541350d4cdaad22ef462d475162f0b5361dac9badf1df7fcbfbf37"}
Jul 10 14:08:50.921628 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.921595    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal" event={"ID":"9b301e1fb865033315caa9abfb1e6ef6","Type":"ContainerStarted","Data":"ca93b7adcac1343d7f13e448634e089d215e394956456a051182198dc03efe31"}
Jul 10 14:08:50.954627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfmxm\" (UniqueName: \"kubernetes.io/projected/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-kube-api-access-kfmxm\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.954627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954521    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g5z8\" (UniqueName: \"kubernetes.io/projected/fb6959d7-ed34-4fe4-9897-12ea79409901-kube-api-access-7g5z8\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.954627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954551    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:50.954627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954577    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cae3993c-e09a-4c84-8d16-6eed8dff45dd-agent-certs\") pod \"konnectivity-agent-5j7xb\" (UID: \"cae3993c-e09a-4c84-8d16-6eed8dff45dd\") " pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:50.954627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954601    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-os-release\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.954627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954628    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-serviceca\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954654    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-socket-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954688    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954717    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954741    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-etc-selinux\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954782    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-host\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954796    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-os-release\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954806    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-registration-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954832    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954861    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954892    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwg7s\" (UniqueName: \"kubernetes.io/projected/2b40128e-bfe7-46ea-baa5-edd38a752eb7-kube-api-access-qwg7s\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954917    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cae3993c-e09a-4c84-8d16-6eed8dff45dd-konnectivity-ca\") pod \"konnectivity-agent-5j7xb\" (UID: \"cae3993c-e09a-4c84-8d16-6eed8dff45dd\") " pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954932    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.954979    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cnibin\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955032    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-host\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.955093 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955049    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmm46\" (UniqueName: \"kubernetes.io/projected/cf0867b7-2f9c-4963-9a99-2c02c22e6267-kube-api-access-tmm46\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955134    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-socket-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955150    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-registration-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955233    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-system-cni-dir\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955251    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-etc-selinux\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-device-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955310    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-system-cni-dir\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955323    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-sys-fs\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955400    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-sys-fs\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955461    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fb6959d7-ed34-4fe4-9897-12ea79409901-device-dir\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.955555    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955559    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cnibin\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.955619    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs podName:2b40128e-bfe7-46ea-baa5-edd38a752eb7 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:51.455600604 +0000 UTC m=+3.081326524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs") pod "network-metrics-daemon-qmqv2" (UID: "2b40128e-bfe7-46ea-baa5-edd38a752eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955639    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cae3993c-e09a-4c84-8d16-6eed8dff45dd-konnectivity-ca\") pod \"konnectivity-agent-5j7xb\" (UID: \"cae3993c-e09a-4c84-8d16-6eed8dff45dd\") " pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955670    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf0867b7-2f9c-4963-9a99-2c02c22e6267-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955706    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cni-binary-copy\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.955950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.955862    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-serviceca\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.956721 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.956179    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.956721 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.956290    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf0867b7-2f9c-4963-9a99-2c02c22e6267-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.958866 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.958841    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cae3993c-e09a-4c84-8d16-6eed8dff45dd-agent-certs\") pod \"konnectivity-agent-5j7xb\" (UID: \"cae3993c-e09a-4c84-8d16-6eed8dff45dd\") " pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:50.965424 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.965343    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:08:50.965424 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.965372    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:08:50.965424 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.965387    2584 projected.go:196] Error preparing data for projected volume kube-api-access-jbpfc for pod openshift-network-diagnostics/network-check-target-5trbg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:50.965639 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:50.965452    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc podName:cba8fa69-2989-4577-b3dd-bdccc1e02ce0 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:51.465433369 +0000 UTC m=+3.091159283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jbpfc" (UniqueName: "kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc") pod "network-check-target-5trbg" (UID: "cba8fa69-2989-4577-b3dd-bdccc1e02ce0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:50.968589 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.968559    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfmxm\" (UniqueName: \"kubernetes.io/projected/d918dd0c-1b5a-49be-b8cf-fe2049a2efac-kube-api-access-kfmxm\") pod \"node-ca-qjk28\" (UID: \"d918dd0c-1b5a-49be-b8cf-fe2049a2efac\") " pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:50.968695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.968572    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwg7s\" (UniqueName: \"kubernetes.io/projected/2b40128e-bfe7-46ea-baa5-edd38a752eb7-kube-api-access-qwg7s\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:50.969209 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.969141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmm46\" (UniqueName: \"kubernetes.io/projected/cf0867b7-2f9c-4963-9a99-2c02c22e6267-kube-api-access-tmm46\") pod \"multus-additional-cni-plugins-q6kjx\" (UID: \"cf0867b7-2f9c-4963-9a99-2c02c22e6267\") " pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:50.970037 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:50.970013    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g5z8\" (UniqueName: \"kubernetes.io/projected/fb6959d7-ed34-4fe4-9897-12ea79409901-kube-api-access-7g5z8\") pod \"aws-ebs-csi-driver-node-rlggg\" (UID: \"fb6959d7-ed34-4fe4-9897-12ea79409901\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:51.040884 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.040846    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t2kz9"
Jul 10 14:08:51.051888 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.051864    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:08:51.052428 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:51.052396    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72c716d_8ba2_4fdf_b1dc_ecfcbb0e1e5d.slice/crio-09075797c3bb0706ab292de11e68880ca0d59ed0e564e94f30482bb8c1f5ae6c WatchSource:0}: Error finding container 09075797c3bb0706ab292de11e68880ca0d59ed0e564e94f30482bb8c1f5ae6c: Status 404 returned error can't find the container with id 09075797c3bb0706ab292de11e68880ca0d59ed0e564e94f30482bb8c1f5ae6c
Jul 10 14:08:51.060994 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.060788    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jl5q6"
Jul 10 14:08:51.062852 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:51.062821    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597072ba_b1e2_4d2e_baa8_a253fefb6284.slice/crio-877d41a523a00bac37351b3005ac0d1e1c45d5ce07ae295057fe2301549e4c3d WatchSource:0}: Error finding container 877d41a523a00bac37351b3005ac0d1e1c45d5ce07ae295057fe2301549e4c3d: Status 404 returned error can't find the container with id 877d41a523a00bac37351b3005ac0d1e1c45d5ce07ae295057fe2301549e4c3d
Jul 10 14:08:51.066016 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.065851    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p4wm7"
Jul 10 14:08:51.071204 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:51.071178    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da23668_9375_4dd5_9332_e7e047d0bd7c.slice/crio-15590be625e110503b1a3190fb5436a46c77cadb53acc6ac02a1336cd5b7e3f9 WatchSource:0}: Error finding container 15590be625e110503b1a3190fb5436a46c77cadb53acc6ac02a1336cd5b7e3f9: Status 404 returned error can't find the container with id 15590be625e110503b1a3190fb5436a46c77cadb53acc6ac02a1336cd5b7e3f9
Jul 10 14:08:51.074297 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.074261    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sjvb7"
Jul 10 14:08:51.076421 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:51.076360    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7df77de_0042_4de4_8622_715fd060430e.slice/crio-90d010f37e2f7d6a7ddf5b657eb79dd01b7481236abeff96d2b7b4f6c44f8e20 WatchSource:0}: Error finding container 90d010f37e2f7d6a7ddf5b657eb79dd01b7481236abeff96d2b7b4f6c44f8e20: Status 404 returned error can't find the container with id 90d010f37e2f7d6a7ddf5b657eb79dd01b7481236abeff96d2b7b4f6c44f8e20
Jul 10 14:08:51.082110 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.082084    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qjk28"
Jul 10 14:08:51.089712 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.089690    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:08:51.091893 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:51.091865    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd918dd0c_1b5a_49be_b8cf_fe2049a2efac.slice/crio-557582ab07b5a5360f374c56f1a675342d5565bfbf7d6b0d3544a803b605bebd WatchSource:0}: Error finding container 557582ab07b5a5360f374c56f1a675342d5565bfbf7d6b0d3544a803b605bebd: Status 404 returned error can't find the container with id 557582ab07b5a5360f374c56f1a675342d5565bfbf7d6b0d3544a803b605bebd
Jul 10 14:08:51.095841 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.095822    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg"
Jul 10 14:08:51.102373 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.102351    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q6kjx"
Jul 10 14:08:51.103025 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:51.102994    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae3993c_e09a_4c84_8d16_6eed8dff45dd.slice/crio-77062d256318bccd17de764442491d48decd65cec3ba40724bad5356aac190df WatchSource:0}: Error finding container 77062d256318bccd17de764442491d48decd65cec3ba40724bad5356aac190df: Status 404 returned error can't find the container with id 77062d256318bccd17de764442491d48decd65cec3ba40724bad5356aac190df
Jul 10 14:08:51.110217 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:08:51.110191    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6959d7_ed34_4fe4_9897_12ea79409901.slice/crio-640fa1a5ebc5bb9867127b948507d3bfef43671876267f2d2df1081180ae54af WatchSource:0}: Error finding container 640fa1a5ebc5bb9867127b948507d3bfef43671876267f2d2df1081180ae54af: Status 404 returned error can't find the container with id 640fa1a5ebc5bb9867127b948507d3bfef43671876267f2d2df1081180ae54af
Jul 10 14:08:51.243646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.243400    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 14:08:51.358971 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.358933    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:51.359136 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.359085    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:51.359208 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.359156    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret podName:259a5382-782c-4cba-8bee-effd8d3b49fc nodeName:}" failed. No retries permitted until 2026-07-10 14:08:52.359136466 +0000 UTC m=+3.984862382 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret") pod "global-pull-secret-syncer-cm2np" (UID: "259a5382-782c-4cba-8bee-effd8d3b49fc") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:51.460294 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.460260    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:51.460484 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.460433    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:51.460545 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.460516    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs podName:2b40128e-bfe7-46ea-baa5-edd38a752eb7 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:52.460495239 +0000 UTC m=+4.086221150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs") pod "network-metrics-daemon-qmqv2" (UID: "2b40128e-bfe7-46ea-baa5-edd38a752eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:51.561318 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.561241    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:51.561544 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.561379    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:08:51.561544 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.561400    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:08:51.561544 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.561410    2584 projected.go:196] Error preparing data for projected volume kube-api-access-jbpfc for pod openshift-network-diagnostics/network-check-target-5trbg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:51.561544 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.561479    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc podName:cba8fa69-2989-4577-b3dd-bdccc1e02ce0 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:52.56145364 +0000 UTC m=+4.187179552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbpfc" (UniqueName: "kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc") pod "network-check-target-5trbg" (UID: "cba8fa69-2989-4577-b3dd-bdccc1e02ce0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:51.809662 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.809615    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:03:51 +0000 UTC" deadline="2027-12-27 04:47:36.523414306 +0000 UTC"
Jul 10 14:08:51.809662 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.809657    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12830h38m44.713762698s"
Jul 10 14:08:51.914783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.914698    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:51.914950 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:51.914846    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:08:51.924395 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.924361    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg" event={"ID":"fb6959d7-ed34-4fe4-9897-12ea79409901","Type":"ContainerStarted","Data":"640fa1a5ebc5bb9867127b948507d3bfef43671876267f2d2df1081180ae54af"}
Jul 10 14:08:51.925742 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.925706    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5j7xb" event={"ID":"cae3993c-e09a-4c84-8d16-6eed8dff45dd","Type":"ContainerStarted","Data":"77062d256318bccd17de764442491d48decd65cec3ba40724bad5356aac190df"}
Jul 10 14:08:51.927043 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.927014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sjvb7" event={"ID":"887b43ac-8a5f-44e7-8f3b-d90ccd70225b","Type":"ContainerStarted","Data":"f355b74393a9cb4a28abbd87e8e39f12266996cf384e74c96358004833de7738"}
Jul 10 14:08:51.928250 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.928228    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jl5q6" event={"ID":"3da23668-9375-4dd5-9332-e7e047d0bd7c","Type":"ContainerStarted","Data":"15590be625e110503b1a3190fb5436a46c77cadb53acc6ac02a1336cd5b7e3f9"}
Jul 10 14:08:51.929581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.929554    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"877d41a523a00bac37351b3005ac0d1e1c45d5ce07ae295057fe2301549e4c3d"}
Jul 10 14:08:51.930853 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.930830    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerStarted","Data":"f662aa7d3c6ff8e16d3cb98a08878100a199d5386d290ba6dc12ad5129e35a70"}
Jul 10 14:08:51.932327 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.932300    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qjk28" event={"ID":"d918dd0c-1b5a-49be-b8cf-fe2049a2efac","Type":"ContainerStarted","Data":"557582ab07b5a5360f374c56f1a675342d5565bfbf7d6b0d3544a803b605bebd"}
Jul 10 14:08:51.933589 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.933563    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p4wm7" event={"ID":"e7df77de-0042-4de4-8622-715fd060430e","Type":"ContainerStarted","Data":"90d010f37e2f7d6a7ddf5b657eb79dd01b7481236abeff96d2b7b4f6c44f8e20"}
Jul 10 14:08:51.935090 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:51.935056    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t2kz9" event={"ID":"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d","Type":"ContainerStarted","Data":"09075797c3bb0706ab292de11e68880ca0d59ed0e564e94f30482bb8c1f5ae6c"}
Jul 10 14:08:52.369346 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.369250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:52.369518 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.369424    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:52.369518 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.369494    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret podName:259a5382-782c-4cba-8bee-effd8d3b49fc nodeName:}" failed. No retries permitted until 2026-07-10 14:08:54.369473996 +0000 UTC m=+5.995199914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret") pod "global-pull-secret-syncer-cm2np" (UID: "259a5382-782c-4cba-8bee-effd8d3b49fc") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:52.470320 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.470275    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:52.470551 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.470527    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:52.470620 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.470610    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs podName:2b40128e-bfe7-46ea-baa5-edd38a752eb7 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:54.470588039 +0000 UTC m=+6.096313972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs") pod "network-metrics-daemon-qmqv2" (UID: "2b40128e-bfe7-46ea-baa5-edd38a752eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:52.571029 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.570985    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:52.571235 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.571217    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:08:52.571339 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.571243    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:08:52.571339 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.571257    2584 projected.go:196] Error preparing data for projected volume kube-api-access-jbpfc for pod openshift-network-diagnostics/network-check-target-5trbg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:52.571339 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.571326    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc podName:cba8fa69-2989-4577-b3dd-bdccc1e02ce0 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:54.571306043 +0000 UTC m=+6.197031959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbpfc" (UniqueName: "kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc") pod "network-check-target-5trbg" (UID: "cba8fa69-2989-4577-b3dd-bdccc1e02ce0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:52.809867 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.809805    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:03:51 +0000 UTC" deadline="2028-03-13 14:49:33.088096415 +0000 UTC"
Jul 10 14:08:52.810253 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.809869    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14688h40m40.278231242s"
Jul 10 14:08:52.917630 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.917596    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:52.917779 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.917741    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:08:52.918196 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.918174    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:52.918307 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:52.918283    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:08:52.944687 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.944651    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal" event={"ID":"7490a82eb930c707bbdadeb01bf134d0","Type":"ContainerStarted","Data":"2924695df097bbb0c700a7d586267d8800989ce3fc2c9a03d09ddd97a3437299"}
Jul 10 14:08:52.966206 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:52.965696    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-123.ec2.internal" podStartSLOduration=2.96567507 podStartE2EDuration="2.96567507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:08:52.965017777 +0000 UTC m=+4.590743711" watchObservedRunningTime="2026-07-10 14:08:52.96567507 +0000 UTC m=+4.591401006"
Jul 10 14:08:53.914882 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:53.914844    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:53.915389 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:53.914993    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:08:53.957868 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:53.957833    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal" event={"ID":"9b301e1fb865033315caa9abfb1e6ef6","Type":"ContainerStarted","Data":"0ad4336058f42e99c02cdf649c0ac16a80f71bc4e66c289e848eb4ce0dd2a7e6"}
Jul 10 14:08:54.385287 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:54.385191    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:54.385463 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.385345    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:54.385463 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.385406    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret podName:259a5382-782c-4cba-8bee-effd8d3b49fc nodeName:}" failed. No retries permitted until 2026-07-10 14:08:58.385387852 +0000 UTC m=+10.011113769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret") pod "global-pull-secret-syncer-cm2np" (UID: "259a5382-782c-4cba-8bee-effd8d3b49fc") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:54.485751 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:54.485708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:54.485981 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.485961    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:54.486071 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.486036    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs podName:2b40128e-bfe7-46ea-baa5-edd38a752eb7 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:58.486014511 +0000 UTC m=+10.111740428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs") pod "network-metrics-daemon-qmqv2" (UID: "2b40128e-bfe7-46ea-baa5-edd38a752eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:54.587057 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:54.587021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:54.587246 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.587180    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:08:54.587246 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.587204    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:08:54.587246 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.587217    2584 projected.go:196] Error preparing data for projected volume kube-api-access-jbpfc for pod openshift-network-diagnostics/network-check-target-5trbg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:54.587405 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.587275    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc podName:cba8fa69-2989-4577-b3dd-bdccc1e02ce0 nodeName:}" failed. No retries permitted until 2026-07-10 14:08:58.587256951 +0000 UTC m=+10.212982886 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbpfc" (UniqueName: "kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc") pod "network-check-target-5trbg" (UID: "cba8fa69-2989-4577-b3dd-bdccc1e02ce0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:54.915847 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:54.915820    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:54.915847 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:54.915847    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:54.916341 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.915964    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:08:54.916341 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:54.916108    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:08:55.914515 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:55.914478    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:55.914698 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:55.914614    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:08:55.965360 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:55.964836    2584 generic.go:356] "Generic (PLEG): container finished" podID="9b301e1fb865033315caa9abfb1e6ef6" containerID="0ad4336058f42e99c02cdf649c0ac16a80f71bc4e66c289e848eb4ce0dd2a7e6" exitCode=0
Jul 10 14:08:55.965360 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:55.964887    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal" event={"ID":"9b301e1fb865033315caa9abfb1e6ef6","Type":"ContainerDied","Data":"0ad4336058f42e99c02cdf649c0ac16a80f71bc4e66c289e848eb4ce0dd2a7e6"}
Jul 10 14:08:56.915010 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:56.914567    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:56.915010 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:56.914802    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:56.915010 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:56.914900    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:08:56.915277 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:56.914708    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:08:57.915205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:57.915169    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:57.915646 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:57.915338    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:08:58.420928 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:58.420895    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:58.421111 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.421032    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:58.421111 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.421101    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret podName:259a5382-782c-4cba-8bee-effd8d3b49fc nodeName:}" failed. No retries permitted until 2026-07-10 14:09:06.42108206 +0000 UTC m=+18.046807970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret") pod "global-pull-secret-syncer-cm2np" (UID: "259a5382-782c-4cba-8bee-effd8d3b49fc") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:08:58.521478 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:58.521437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:58.521637 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.521608    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:58.521714 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.521689    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs podName:2b40128e-bfe7-46ea-baa5-edd38a752eb7 nodeName:}" failed. No retries permitted until 2026-07-10 14:09:06.521668357 +0000 UTC m=+18.147394272 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs") pod "network-metrics-daemon-qmqv2" (UID: "2b40128e-bfe7-46ea-baa5-edd38a752eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:08:58.622549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:58.622503    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:58.622748 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.622648    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:08:58.622748 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.622669    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:08:58.622748 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.622679    2584 projected.go:196] Error preparing data for projected volume kube-api-access-jbpfc for pod openshift-network-diagnostics/network-check-target-5trbg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:58.622914 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.622754    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc podName:cba8fa69-2989-4577-b3dd-bdccc1e02ce0 nodeName:}" failed. No retries permitted until 2026-07-10 14:09:06.622735299 +0000 UTC m=+18.248461231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbpfc" (UniqueName: "kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc") pod "network-check-target-5trbg" (UID: "cba8fa69-2989-4577-b3dd-bdccc1e02ce0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:08:58.918797 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:58.918417    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:08:58.918797 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:58.918556    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:08:58.918797 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.918667    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:08:58.919264 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:58.918848    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:08:59.914473 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:08:59.914424    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:08:59.914781 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:08:59.914559    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:00.915411 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:00.915380    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:00.915900 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:00.915520    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:00.915900 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:00.915526    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:00.915900 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:00.915614    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:01.914667 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.914448    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:01.914819 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:01.914783    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:01.979497 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.979469    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg" event={"ID":"fb6959d7-ed34-4fe4-9897-12ea79409901","Type":"ContainerStarted","Data":"a9115ea35fffd5e42a968d54840bad228c7abbaf52ce4ee1609e219ec4a9e23c"}
Jul 10 14:09:01.980971 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.980916    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5j7xb" event={"ID":"cae3993c-e09a-4c84-8d16-6eed8dff45dd","Type":"ContainerStarted","Data":"8a6e7c00a4ef2f260b9c261445a84d4121c595260c3370251e87cf6a2098f72d"}
Jul 10 14:09:01.982695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.982609    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jl5q6" event={"ID":"3da23668-9375-4dd5-9332-e7e047d0bd7c","Type":"ContainerStarted","Data":"725d978b62eca3ebaa03188999c74f19df6a950db82f1e1fb06cca6317ee53c3"}
Jul 10 14:09:01.985508 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.985484    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal" event={"ID":"9b301e1fb865033315caa9abfb1e6ef6","Type":"ContainerStarted","Data":"36426224df1c112f501fbde46ffb7d6ec7bbe6b5209bff703b2783fd1289d897"}
Jul 10 14:09:01.987753 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.987452    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerStarted","Data":"a617768e831860721b0df16c18fbd44dc1408256b73bd9b3e41c77cf2bd0e00e"}
Jul 10 14:09:01.989039 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.988993    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qjk28" event={"ID":"d918dd0c-1b5a-49be-b8cf-fe2049a2efac","Type":"ContainerStarted","Data":"4bb848303df5a00b108aa23083fed58b5df9d62b824236d09b52ac60894b3eec"}
Jul 10 14:09:01.990464 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.990424    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t2kz9" event={"ID":"b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d","Type":"ContainerStarted","Data":"53698f38e61ebb6ec0de494c6f289dd7ba49755d2d0668a92c53b27e7d7a57ad"}
Jul 10 14:09:01.998536 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:01.998497    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5j7xb" podStartSLOduration=2.506081694 podStartE2EDuration="12.99848474s" totalImagesPullingTime="10.492403046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:01.997665277 +0000 UTC m=+13.623391207" watchObservedRunningTime="2026-07-10 14:09:01.99848474 +0000 UTC m=+13.624210671"
Jul 10 14:09:02.020860 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.020828    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:02.038852 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.038755    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-123.ec2.internal" podStartSLOduration=12.038711652 podStartE2EDuration="12.038711652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:02.038553554 +0000 UTC m=+13.664279485" watchObservedRunningTime="2026-07-10 14:09:02.038711652 +0000 UTC m=+13.664437583"
Jul 10 14:09:02.056234 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.056158    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t2kz9" podStartSLOduration=2.532764403 podStartE2EDuration="13.056140117s" totalImagesPullingTime="10.523375714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:02.055264447 +0000 UTC m=+13.680990378" watchObservedRunningTime="2026-07-10 14:09:02.056140117 +0000 UTC m=+13.681866049"
Jul 10 14:09:02.086399 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.086358    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jl5q6" podStartSLOduration=2.541992251 podStartE2EDuration="13.086345504s" totalImagesPullingTime="10.544353253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:02.072293677 +0000 UTC m=+13.698019608" watchObservedRunningTime="2026-07-10 14:09:02.086345504 +0000 UTC m=+13.712071435"
Jul 10 14:09:02.086637 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.086600    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qjk28" podStartSLOduration=2.582972186 podStartE2EDuration="13.086594037s" totalImagesPullingTime="10.503621851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:02.086095763 +0000 UTC m=+13.711821693" watchObservedRunningTime="2026-07-10 14:09:02.086594037 +0000 UTC m=+13.712319991"
Jul 10 14:09:02.238110 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.238062    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:02.914968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.914768    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:02.914968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.914791    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:02.914968 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:02.914920    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:02.915244 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:02.915039    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:02.994272 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.994236    2584 generic.go:356] "Generic (PLEG): container finished" podID="cf0867b7-2f9c-4963-9a99-2c02c22e6267" containerID="a617768e831860721b0df16c18fbd44dc1408256b73bd9b3e41c77cf2bd0e00e" exitCode=0
Jul 10 14:09:02.994901 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:02.994336    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerDied","Data":"a617768e831860721b0df16c18fbd44dc1408256b73bd9b3e41c77cf2bd0e00e"}
Jul 10 14:09:03.007501 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:03.007471    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:03.609321 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:03.609284    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:03.807510 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:03.807469    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:03.914864 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:03.914767    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:03.915020 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:03.914897    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:03.998587 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:03.998538    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sjvb7" event={"ID":"887b43ac-8a5f-44e7-8f3b-d90ccd70225b","Type":"ContainerStarted","Data":"ca183a0774227ec4b60bd346ddb2c2f20c82cc6bd164ed47f6ef95a35575041f"}
Jul 10 14:09:04.015304 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:04.015248    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sjvb7" podStartSLOduration=4.501358152 podStartE2EDuration="15.015227657s" totalImagesPullingTime="10.513869505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:04.01473688 +0000 UTC m=+15.640462807" watchObservedRunningTime="2026-07-10 14:09:04.015227657 +0000 UTC m=+15.640953592"
Jul 10 14:09:04.208873 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:04.208794    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:04.404089 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:04.404056    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:04.914679 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:04.914640    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:04.914679 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:04.914658    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:04.914941 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:04.914798    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:04.914941 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:04.914901    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:05.013578 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:05.013535    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:05.072009 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:05.071963    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:05.411155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:05.411102    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:05.470254 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:05.470135    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:09:05.604128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:05.604094    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:05.915251 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:05.915222    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:05.915418 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:05.915328    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:06.024994 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.024963    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.181912 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.181814    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.282514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.282449    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.351431 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.351385    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.438599 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.438506    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.484473 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.484441    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:06.484638 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.484580    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:09:06.484638 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.484634    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret podName:259a5382-782c-4cba-8bee-effd8d3b49fc nodeName:}" failed. No retries permitted until 2026-07-10 14:09:22.484620387 +0000 UTC m=+34.110346297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret") pod "global-pull-secret-syncer-cm2np" (UID: "259a5382-782c-4cba-8bee-effd8d3b49fc") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:09:06.506929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.506810    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.584834 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.584803    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:06.584990 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.584952    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:09:06.585030 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.585015    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs podName:2b40128e-bfe7-46ea-baa5-edd38a752eb7 nodeName:}" failed. No retries permitted until 2026-07-10 14:09:22.584998387 +0000 UTC m=+34.210724297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs") pod "network-metrics-daemon-qmqv2" (UID: "2b40128e-bfe7-46ea-baa5-edd38a752eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:09:06.617032 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.616851    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.685895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.685865    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:06.686065 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.686004    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:09:06.686065 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.686021    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:09:06.686065 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.686030    2584 projected.go:196] Error preparing data for projected volume kube-api-access-jbpfc for pod openshift-network-diagnostics/network-check-target-5trbg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:09:06.686185 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.686084    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc podName:cba8fa69-2989-4577-b3dd-bdccc1e02ce0 nodeName:}" failed. No retries permitted until 2026-07-10 14:09:22.686067923 +0000 UTC m=+34.311793835 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbpfc" (UniqueName: "kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc") pod "network-check-target-5trbg" (UID: "cba8fa69-2989-4577-b3dd-bdccc1e02ce0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:09:06.722245 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.722141    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.820664 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.820633    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:09:06.821407 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.821389    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:09:06.872059 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.872032    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:06.915272 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.915213    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:06.915381 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.915359    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:06.915654 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.915530    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:06.915760 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:06.915659    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:06.933206 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:06.933175    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:07.008458 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.008417    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg" event={"ID":"fb6959d7-ed34-4fe4-9897-12ea79409901","Type":"ContainerStarted","Data":"02b75399224fe4c4863283f41d82f5e98a980925fe7f22cb5c4f5d45e51b9734"}
Jul 10 14:09:07.009327 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.009307    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5j7xb"
Jul 10 14:09:07.010512 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.010486    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 14:09:07.245704 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.245614    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:07.805856 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.805744    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T14:09:07.010506882Z","UUID":"e39223d7-67ce-426b-9a10-4792b0e46226","Handler":null,"Name":"","Endpoint":""}
Jul 10 14:09:07.807568 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.807541    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 14:09:07.807702 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.807603    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 14:09:07.870409 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.870371    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:07.914534 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:07.914501    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:07.914678 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:07.914622    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:08.012232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:08.012183    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p4wm7" event={"ID":"e7df77de-0042-4de4-8622-715fd060430e","Type":"ContainerStarted","Data":"71fd2a5af182f02a0e3c74bb09f421eb76d4253df298526039d8ce025c5c017c"}
Jul 10 14:09:08.097447 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:08.097410    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:08.292688 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:08.292660    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:08.392927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:08.392895    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:08.493924 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:08.493883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:08.916319 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:08.916285    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:08.916496 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:08.916424    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:08.916496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:08.916484    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:08.916631 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:08.916594    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:09.017430 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:09.017392    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg" event={"ID":"fb6959d7-ed34-4fe4-9897-12ea79409901","Type":"ContainerStarted","Data":"9e79f03cb5ef7d011a800e01acd04080008262d91d7d3a3519527b48a2bfe026"}
Jul 10 14:09:09.025418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:09.025365    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:09.037915 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:09.037862    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rlggg" podStartSLOduration=3.270915178 podStartE2EDuration="20.037840008s" totalImagesPullingTime="16.76692483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:09.03751311 +0000 UTC m=+20.663239043" watchObservedRunningTime="2026-07-10 14:09:09.037840008 +0000 UTC m=+20.663565966"
Jul 10 14:09:09.038103 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:09.038062    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p4wm7" podStartSLOduration=4.070366823 podStartE2EDuration="20.038052292s" totalImagesPullingTime="15.967685469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:08.056200434 +0000 UTC m=+19.681926380" watchObservedRunningTime="2026-07-10 14:09:09.038052292 +0000 UTC m=+20.663778227"
Jul 10 14:09:09.796041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:09.796005    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:09.915145 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:09.915097    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:09.915334 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:09.915244    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:10.915372 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:10.915338    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:10.915798 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:10.915339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:10.915798 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:10.915467    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:10.915798 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:10.915528    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:11.023161 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:11.023128    2584 generic.go:356] "Generic (PLEG): container finished" podID="cf0867b7-2f9c-4963-9a99-2c02c22e6267" containerID="b45402382bac82001e666f5b160cd90d4ecdf2a99f4c53bee8fb959647b40049" exitCode=0
Jul 10 14:09:11.023315 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:11.023178    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerDied","Data":"b45402382bac82001e666f5b160cd90d4ecdf2a99f4c53bee8fb959647b40049"}
Jul 10 14:09:11.915373 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:11.915346    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:11.915926 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:11.915465    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:12.146035 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:12.146002    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:12.244732 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:12.244635    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:12.344633 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:12.344597    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:12.671232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:12.671196    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:12.914616 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:12.914588    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:12.914796 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:12.914744    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:12.914881 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:12.914741    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:12.914881 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:12.914831    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:13.914532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:13.914496    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:13.914944 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:13.914651    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:14.914814 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:14.914781    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:14.915228 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:14.914780    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:14.915228 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:14.914926    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:14.915228 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:14.915025    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:15.915052 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:15.915021    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:15.915434 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:15.915153    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:16.035535 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.035298    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"19fc38e084cf30e202f33056b460df9a58de6efbb67cbcf9eb7eae660955b6bf"}
Jul 10 14:09:16.317457 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.317421    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:16.475248 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.475226    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:16.574700 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.574674    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:16.771710 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.771683    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:16.849566 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.849536    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:16.914872 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.914842    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:16.915015 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:16.914842    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:16.915015 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:16.914948    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:16.915106 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:16.915028    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:17.039573 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.039504    2584 generic.go:356] "Generic (PLEG): container finished" podID="cf0867b7-2f9c-4963-9a99-2c02c22e6267" containerID="e09522be7639c23f3aa79b45fbaf510b253addb1be864f987d0f7c55433adcb7" exitCode=0
Jul 10 14:09:17.039710 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.039590    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerDied","Data":"e09522be7639c23f3aa79b45fbaf510b253addb1be864f987d0f7c55433adcb7"}
Jul 10 14:09:17.042369 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.042353    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:09:17.042742 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.042705    2584 generic.go:356] "Generic (PLEG): container finished" podID="597072ba-b1e2-4d2e-baa8-a253fefb6284" containerID="0340bfca1ce0cb76bed4de474315822a0931e6f9a986d7a46a60a2dc29b042dc" exitCode=1
Jul 10 14:09:17.042807 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.042751    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"27942623ed20710de023e6ccb9d273511ee15a9e7b18a65f585efc0306c6fd49"}
Jul 10 14:09:17.042807 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.042801    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"a0565fb16e15def43a28af752d1fe789e0ce003e51ecdd79b0a3f5f682307efe"}
Jul 10 14:09:17.042886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.042837    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"c1aeeefea1490e3e703b6f286008e8142788663750e1c4ce6ab9db07f4e86b3e"}
Jul 10 14:09:17.042886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.042865    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"153b41fb8a85c1343ab37d89e15768cdabee0309815b48bef5081da22ecbaf52"}
Jul 10 14:09:17.042955 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.042891    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerDied","Data":"0340bfca1ce0cb76bed4de474315822a0931e6f9a986d7a46a60a2dc29b042dc"}
Jul 10 14:09:17.915498 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:17.915323    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:17.915964 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:17.915574    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:18.915437 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:18.915415    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:18.915556 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:18.915513    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:18.915556 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:18.915553    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:18.915946 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:18.915619    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:19.050219 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:19.050181    2584 generic.go:356] "Generic (PLEG): container finished" podID="cf0867b7-2f9c-4963-9a99-2c02c22e6267" containerID="18af2fd95ef907c0bbfd70cb2422687dc59680f40fface65b350acd29ba539cf" exitCode=0
Jul 10 14:09:19.050364 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:19.050258    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerDied","Data":"18af2fd95ef907c0bbfd70cb2422687dc59680f40fface65b350acd29ba539cf"}
Jul 10 14:09:19.053591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:19.053573    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:09:19.054011 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:19.053984    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"51c34fbe5f35dd4a4df04b6ab4d3a7d6b283a672169853630f351df04f37e37d"}
Jul 10 14:09:19.914504 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:19.914472    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:19.914694 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:19.914578    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:20.385616 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:20.385585    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:20.500073 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:20.500037    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:20.597223 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:20.597192    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:20.915420 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:20.915389    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:20.915600 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:20.915527    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:20.915600 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:20.915566    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:20.915714 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:20.915668    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:21.914534 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:21.914506    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:21.914966 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:21.914603    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:22.064128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.064100    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:09:22.064544 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.064505    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"d055c2fc4c6eae19609ac1359d9d46cfda8833d7b47805f9ac5478741ef2bbd6"}
Jul 10 14:09:22.064774 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.064752    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:09:22.065192 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.065166    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:09:22.065332 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.065306    2584 scope.go:117] "RemoveContainer" containerID="0340bfca1ce0cb76bed4de474315822a0931e6f9a986d7a46a60a2dc29b042dc"
Jul 10 14:09:22.082407 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.082389    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:09:22.500044 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.500000    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:22.500235 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.500177    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 14:09:22.500557 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.500241    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret podName:259a5382-782c-4cba-8bee-effd8d3b49fc nodeName:}" failed. No retries permitted until 2026-07-10 14:09:54.500221807 +0000 UTC m=+66.125947720 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret") pod "global-pull-secret-syncer-cm2np" (UID: "259a5382-782c-4cba-8bee-effd8d3b49fc") : object "kube-system"/"original-pull-secret" not registered
Jul 10 14:09:22.510770 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.510695    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:22.567579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.567544    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:22.600664 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.600623    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:22.600857 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.600793    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:09:22.600920 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.600870    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs podName:2b40128e-bfe7-46ea-baa5-edd38a752eb7 nodeName:}" failed. No retries permitted until 2026-07-10 14:09:54.600848872 +0000 UTC m=+66.226574804 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs") pod "network-metrics-daemon-qmqv2" (UID: "2b40128e-bfe7-46ea-baa5-edd38a752eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 14:09:22.701505 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.701301    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:22.701646 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.701477    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 14:09:22.701646 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.701578    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 14:09:22.701646 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.701591    2584 projected.go:196] Error preparing data for projected volume kube-api-access-jbpfc for pod openshift-network-diagnostics/network-check-target-5trbg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:09:22.701813 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.701655    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc podName:cba8fa69-2989-4577-b3dd-bdccc1e02ce0 nodeName:}" failed. No retries permitted until 2026-07-10 14:09:54.701636893 +0000 UTC m=+66.327362806 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbpfc" (UniqueName: "kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc") pod "network-check-target-5trbg" (UID: "cba8fa69-2989-4577-b3dd-bdccc1e02ce0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 14:09:22.915285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.915258    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:22.915648 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:22.915264    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:22.915648 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.915362    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:22.915648 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:22.915478    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:23.069449 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.069425    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:09:23.069768 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.069748    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" event={"ID":"597072ba-b1e2-4d2e-baa8-a253fefb6284","Type":"ContainerStarted","Data":"571d49f5680e0935e0598a0645658f0e8a3c2d9ff4a3ba2e746701d5912e10a7"}
Jul 10 14:09:23.069956 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.069940    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:09:23.084692 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.084670    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:09:23.099309 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.099270    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t" podStartSLOduration=9.283806622 podStartE2EDuration="34.099254771s" totalImagesPullingTime="24.815448149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:23.098765327 +0000 UTC m=+34.724491257" watchObservedRunningTime="2026-07-10 14:09:23.099254771 +0000 UTC m=+34.724980703"
Jul 10 14:09:23.399368 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.399330    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cm2np"]
Jul 10 14:09:23.399530 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.399499    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:23.399632 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:23.399606    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:23.400140 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.400114    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5trbg"]
Jul 10 14:09:23.400237 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.400226    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:23.400374 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:23.400351    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:23.401420 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.401399    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qmqv2"]
Jul 10 14:09:23.401507 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.401488    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:23.401604 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:23.401583    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:23.983909 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:23.983875    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:24.083384 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:24.083349    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:24.494624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:24.494581    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:24.914808 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:24.914774    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:24.914984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:24.914773    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:24.914984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:24.914780    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:24.914984 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:24.914928    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:24.915113 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:24.915052    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:24.915168 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:24.915148    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:26.079064 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:26.079036    2584 generic.go:356] "Generic (PLEG): container finished" podID="cf0867b7-2f9c-4963-9a99-2c02c22e6267" containerID="894285298a00b6f3dad87c584777c50df254b5746aea9987442bccaea021accf" exitCode=0
Jul 10 14:09:26.079583 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:26.079096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerDied","Data":"894285298a00b6f3dad87c584777c50df254b5746aea9987442bccaea021accf"}
Jul 10 14:09:26.084574 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:26.084546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:26.188006 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:26.187966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:26.914805 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:26.914771    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:26.914972 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:26.914771    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:26.914972 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:26.914900    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:26.914972 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:26.914787    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:26.914972 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:26.914950    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:26.915086 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:26.915020    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:27.084152 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.084117    2584 generic.go:356] "Generic (PLEG): container finished" podID="cf0867b7-2f9c-4963-9a99-2c02c22e6267" containerID="40a15c3cca1f19baa16359e8513376f6d3a4c4f272dce3064a88dfb07f1eeeeb" exitCode=0
Jul 10 14:09:27.084503 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.084170    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerDied","Data":"40a15c3cca1f19baa16359e8513376f6d3a4c4f272dce3064a88dfb07f1eeeeb"}
Jul 10 14:09:27.573205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.573027    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:27.682155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.682126    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:27.791279 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.791215    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:27.844079 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.844014    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m8tpr"]
Jul 10 14:09:27.845989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.845974    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.848920 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.848896    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 14:09:27.849242 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.849222    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 14:09:27.849242 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.849236    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 14:09:27.850330 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.850312    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 14:09:27.850441 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.850344    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 14:09:27.850441 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.850388    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 14:09:27.850561 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.850502    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tn4nf\""
Jul 10 14:09:27.880391 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.880361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:27.943136 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943112    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-root\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943152    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943168    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqn7r\" (UniqueName: \"kubernetes.io/projected/f3f41fdc-56bd-4be1-915a-6c2a15f84169-kube-api-access-sqn7r\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-tls\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943227    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-sys\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943244    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-textfile\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943257    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-wtmp\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943273    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3f41fdc-56bd-4be1-915a-6c2a15f84169-metrics-client-ca\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.943293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.943290    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:27.945552 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:27.945525    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.043595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043563    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-tls\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043601    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-sys\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043619    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-textfile\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-wtmp\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043655    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3f41fdc-56bd-4be1-915a-6c2a15f84169-metrics-client-ca\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043683    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043696    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-sys\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043784    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-root\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043833    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043833    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-wtmp\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.043892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043859    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqn7r\" (UniqueName: \"kubernetes.io/projected/f3f41fdc-56bd-4be1-915a-6c2a15f84169-kube-api-access-sqn7r\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.044234 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.043905    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f3f41fdc-56bd-4be1-915a-6c2a15f84169-root\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.044234 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.044000    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-textfile\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.044698 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.044674    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3f41fdc-56bd-4be1-915a-6c2a15f84169-metrics-client-ca\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.044822 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.044676    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-accelerators-collector-config\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.047370 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.047338    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.047694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.047673    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-tls\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.047791 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.047657    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f3f41fdc-56bd-4be1-915a-6c2a15f84169-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.052483 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.052463    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqn7r\" (UniqueName: \"kubernetes.io/projected/f3f41fdc-56bd-4be1-915a-6c2a15f84169-kube-api-access-sqn7r\") pod \"node-exporter-m8tpr\" (UID: \"f3f41fdc-56bd-4be1-915a-6c2a15f84169\") " pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.089367 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.089337    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" event={"ID":"cf0867b7-2f9c-4963-9a99-2c02c22e6267","Type":"ContainerStarted","Data":"90f719aaa80b0054de944b4e07ac5150d3e45240e9f62f6644acb9e716972bba"}
Jul 10 14:09:28.112128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.112043    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q6kjx" podStartSLOduration=8.019560631 podStartE2EDuration="39.112029231s" totalImagesPullingTime="31.0924686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:28.111959898 +0000 UTC m=+39.737685829" watchObservedRunningTime="2026-07-10 14:09:28.112029231 +0000 UTC m=+39.737755161"
Jul 10 14:09:28.147354 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.147330    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.155269 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.155245    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m8tpr"
Jul 10 14:09:28.164328 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:28.164298    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f41fdc_56bd_4be1_915a_6c2a15f84169.slice/crio-9da3034b331c15670944decdb4827764e7c390ae5310b3e13e8acb34707d40fd WatchSource:0}: Error finding container 9da3034b331c15670944decdb4827764e7c390ae5310b3e13e8acb34707d40fd: Status 404 returned error can't find the container with id 9da3034b331c15670944decdb4827764e7c390ae5310b3e13e8acb34707d40fd
Jul 10 14:09:28.251007 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.250977    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.635015 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.634963    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.702638 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.702606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.797060 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.797022    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.883053 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.883019    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:28.915863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.915787    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:28.916026 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.915896    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:28.916026 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:28.915924    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cm2np" podUID="259a5382-782c-4cba-8bee-effd8d3b49fc"
Jul 10 14:09:28.916026 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:28.915964    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5trbg" podUID="cba8fa69-2989-4577-b3dd-bdccc1e02ce0"
Jul 10 14:09:28.916026 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:28.916013    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:28.916229 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:28.916103    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qmqv2" podUID="2b40128e-bfe7-46ea-baa5-edd38a752eb7"
Jul 10 14:09:29.004148 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.004112    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:29.093117 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.093083    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8tpr" event={"ID":"f3f41fdc-56bd-4be1-915a-6c2a15f84169","Type":"ContainerStarted","Data":"9da3034b331c15670944decdb4827764e7c390ae5310b3e13e8acb34707d40fd"}
Jul 10 14:09:29.667639 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.667589    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-123.ec2.internal" event="NodeReady"
Jul 10 14:09:29.667812 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.667740    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 14:09:29.717827 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.717746    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:29.717827 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.717782    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hcxqk"]
Jul 10 14:09:29.719659 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.719641    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.720626 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.720598    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kvltq"]
Jul 10 14:09:29.722045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.722032    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:29.722793 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.722773    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gwmhf\""
Jul 10 14:09:29.722889 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.722880    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 14:09:29.722962 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.722949    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 14:09:29.725257 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.725235    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 14:09:29.725339 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.725256    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 14:09:29.725339 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.725272    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 14:09:29.725531 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.725517    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2tvlk\""
Jul 10 14:09:29.735489 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.735469    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hcxqk"]
Jul 10 14:09:29.736005 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.735985    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kvltq"]
Jul 10 14:09:29.749021 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.749001    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fjb47"]
Jul 10 14:09:29.750797 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.750781    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.753565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.753539    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 14:09:29.753668 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.753624    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 14:09:29.753668 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.753660    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 14:09:29.753795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.753678    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-54rbs\""
Jul 10 14:09:29.753795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.753622    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 14:09:29.764926 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.764905    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjb47"]
Jul 10 14:09:29.793533 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.793507    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:29.858435 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858404    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slktv\" (UniqueName: \"kubernetes.io/projected/f45056ea-8aa6-4200-a036-2591f6364b9a-kube-api-access-slktv\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.858565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858441    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89zzc\" (UniqueName: \"kubernetes.io/projected/75c35095-9063-4aa7-a7c8-4e518a91821b-kube-api-access-89zzc\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.858565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858483    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/75c35095-9063-4aa7-a7c8-4e518a91821b-data-volume\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.858565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858508    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f45056ea-8aa6-4200-a036-2591f6364b9a-config-volume\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.858565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858524    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f45056ea-8aa6-4200-a036-2591f6364b9a-metrics-tls\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.858565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858546    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd39c349-df40-4932-92b7-a29e8c11e195-cert\") pod \"ingress-canary-kvltq\" (UID: \"cd39c349-df40-4932-92b7-a29e8c11e195\") " pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:29.858755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858586    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/75c35095-9063-4aa7-a7c8-4e518a91821b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.858755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858623    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45056ea-8aa6-4200-a036-2591f6364b9a-tmp-dir\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.858755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858658    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/75c35095-9063-4aa7-a7c8-4e518a91821b-crio-socket\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.858755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858676    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/75c35095-9063-4aa7-a7c8-4e518a91821b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.858755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.858695    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhggm\" (UniqueName: \"kubernetes.io/projected/cd39c349-df40-4932-92b7-a29e8c11e195-kube-api-access-nhggm\") pod \"ingress-canary-kvltq\" (UID: \"cd39c349-df40-4932-92b7-a29e8c11e195\") " pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:29.893947 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.893912    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:29.959915 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.959888    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/75c35095-9063-4aa7-a7c8-4e518a91821b-crio-socket\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.960017 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.959920    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/75c35095-9063-4aa7-a7c8-4e518a91821b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.960017 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.959940    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhggm\" (UniqueName: \"kubernetes.io/projected/cd39c349-df40-4932-92b7-a29e8c11e195-kube-api-access-nhggm\") pod \"ingress-canary-kvltq\" (UID: \"cd39c349-df40-4932-92b7-a29e8c11e195\") " pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:29.960017 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.959964    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktv\" (UniqueName: \"kubernetes.io/projected/f45056ea-8aa6-4200-a036-2591f6364b9a-kube-api-access-slktv\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.960017 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.959992    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89zzc\" (UniqueName: \"kubernetes.io/projected/75c35095-9063-4aa7-a7c8-4e518a91821b-kube-api-access-89zzc\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.960192 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960026    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/75c35095-9063-4aa7-a7c8-4e518a91821b-data-volume\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.960192 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960136    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/75c35095-9063-4aa7-a7c8-4e518a91821b-crio-socket\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.960192 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960165    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f45056ea-8aa6-4200-a036-2591f6364b9a-config-volume\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.960329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960196    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f45056ea-8aa6-4200-a036-2591f6364b9a-metrics-tls\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.960329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd39c349-df40-4932-92b7-a29e8c11e195-cert\") pod \"ingress-canary-kvltq\" (UID: \"cd39c349-df40-4932-92b7-a29e8c11e195\") " pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:29.960329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960290    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/75c35095-9063-4aa7-a7c8-4e518a91821b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.960329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960327    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45056ea-8aa6-4200-a036-2591f6364b9a-tmp-dir\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.960745 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960702    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/75c35095-9063-4aa7-a7c8-4e518a91821b-data-volume\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.960898 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.960745    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f45056ea-8aa6-4200-a036-2591f6364b9a-tmp-dir\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.961033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.961011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/75c35095-9063-4aa7-a7c8-4e518a91821b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.961090 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.961015    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f45056ea-8aa6-4200-a036-2591f6364b9a-config-volume\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.963591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.963472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/75c35095-9063-4aa7-a7c8-4e518a91821b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.963656 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.963516    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f45056ea-8aa6-4200-a036-2591f6364b9a-metrics-tls\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.963656 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.963565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd39c349-df40-4932-92b7-a29e8c11e195-cert\") pod \"ingress-canary-kvltq\" (UID: \"cd39c349-df40-4932-92b7-a29e8c11e195\") " pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:29.967986 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.967932    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slktv\" (UniqueName: \"kubernetes.io/projected/f45056ea-8aa6-4200-a036-2591f6364b9a-kube-api-access-slktv\") pod \"dns-default-hcxqk\" (UID: \"f45056ea-8aa6-4200-a036-2591f6364b9a\") " pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:29.967986 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.967958    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89zzc\" (UniqueName: \"kubernetes.io/projected/75c35095-9063-4aa7-a7c8-4e518a91821b-kube-api-access-89zzc\") pod \"insights-runtime-extractor-fjb47\" (UID: \"75c35095-9063-4aa7-a7c8-4e518a91821b\") " pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:29.968125 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:29.968053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhggm\" (UniqueName: \"kubernetes.io/projected/cd39c349-df40-4932-92b7-a29e8c11e195-kube-api-access-nhggm\") pod \"ingress-canary-kvltq\" (UID: \"cd39c349-df40-4932-92b7-a29e8c11e195\") " pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:30.029841 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.029809    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:30.034922 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.034905    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kvltq"
Jul 10 14:09:30.059164 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.059152    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjb47"
Jul 10 14:09:30.098079 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.098049    2584 generic.go:356] "Generic (PLEG): container finished" podID="f3f41fdc-56bd-4be1-915a-6c2a15f84169" containerID="b5ecb27048e7839c4b419b023d652e27361319cfaff5ceb4d469ef30aa499f11" exitCode=0
Jul 10 14:09:30.098486 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.098132    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8tpr" event={"ID":"f3f41fdc-56bd-4be1-915a-6c2a15f84169","Type":"ContainerDied","Data":"b5ecb27048e7839c4b419b023d652e27361319cfaff5ceb4d469ef30aa499f11"}
Jul 10 14:09:30.252031 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.252002    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjb47"]
Jul 10 14:09:30.255490 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:30.255460    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c35095_9063_4aa7_a7c8_4e518a91821b.slice/crio-054c0f654d95a72ef0da276c4d1aa5ec9b6e3660faed8013705b745fd50dab34 WatchSource:0}: Error finding container 054c0f654d95a72ef0da276c4d1aa5ec9b6e3660faed8013705b745fd50dab34: Status 404 returned error can't find the container with id 054c0f654d95a72ef0da276c4d1aa5ec9b6e3660faed8013705b745fd50dab34
Jul 10 14:09:30.258756 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.258700    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hcxqk"]
Jul 10 14:09:30.260792 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:30.260770    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45056ea_8aa6_4200_a036_2591f6364b9a.slice/crio-80e41db71ba5dd49ecf1ff6ff84f4fadde3280409b14c68d3efde7ada443aa57 WatchSource:0}: Error finding container 80e41db71ba5dd49ecf1ff6ff84f4fadde3280409b14c68d3efde7ada443aa57: Status 404 returned error can't find the container with id 80e41db71ba5dd49ecf1ff6ff84f4fadde3280409b14c68d3efde7ada443aa57
Jul 10 14:09:30.263628 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.263549    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kvltq"]
Jul 10 14:09:30.267465 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:30.267440    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd39c349_df40_4932_92b7_a29e8c11e195.slice/crio-57b67c42d6b6351a6bada252fff9e4d2ca28249d252b702988d0b24faed0636c WatchSource:0}: Error finding container 57b67c42d6b6351a6bada252fff9e4d2ca28249d252b702988d0b24faed0636c: Status 404 returned error can't find the container with id 57b67c42d6b6351a6bada252fff9e4d2ca28249d252b702988d0b24faed0636c
Jul 10 14:09:30.776311 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.776278    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:30.883693 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.883429    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:30.891934 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.891913    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6845b84675-k9755"]
Jul 10 14:09:30.906294 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.906265    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:30.909336 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.909248    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6845b84675-k9755"]
Jul 10 14:09:30.911205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.909796    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 10 14:09:30.911205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.910046    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9sbj4\""
Jul 10 14:09:30.911205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.910235    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 10 14:09:30.911205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.910473    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 10 14:09:30.911205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.910647    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 10 14:09:30.911205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.910865    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 10 14:09:30.911205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.911045    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d8rj3r9ef9d2f\""
Jul 10 14:09:30.915521 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.915499    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:30.915835 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.915706    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:30.915944 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.915863    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:30.918712 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.918460    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 14:09:30.918880 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.918861    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7ddd8\""
Jul 10 14:09:30.918960 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.918922    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 14:09:30.919034 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.919014    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fptgw\""
Jul 10 14:09:30.919264 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.919242    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 14:09:30.919853 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.919834    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 14:09:30.966355 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:30.966283    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071403    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071453    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-grpc-tls\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071488    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adad8394-cc75-4339-9898-a25835beef4f-metrics-client-ca\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071666    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071704    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071781    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-tls\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.072428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.071825    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxhx5\" (UniqueName: \"kubernetes.io/projected/adad8394-cc75-4339-9898-a25835beef4f-kube-api-access-qxhx5\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.091023 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.090956    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:31.121047 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.120934    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kvltq" event={"ID":"cd39c349-df40-4932-92b7-a29e8c11e195","Type":"ContainerStarted","Data":"57b67c42d6b6351a6bada252fff9e4d2ca28249d252b702988d0b24faed0636c"}
Jul 10 14:09:31.123252 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.123222    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjb47" event={"ID":"75c35095-9063-4aa7-a7c8-4e518a91821b","Type":"ContainerStarted","Data":"054c0f654d95a72ef0da276c4d1aa5ec9b6e3660faed8013705b745fd50dab34"}
Jul 10 14:09:31.126076 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.126049    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8tpr" event={"ID":"f3f41fdc-56bd-4be1-915a-6c2a15f84169","Type":"ContainerStarted","Data":"327682991bffbe6788ab616c2158c4edf9965f6ac7b8ac7401576dbfa34b31ed"}
Jul 10 14:09:31.126155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.126101    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m8tpr" event={"ID":"f3f41fdc-56bd-4be1-915a-6c2a15f84169","Type":"ContainerStarted","Data":"d104df5725a74216a9430464739a4284c3c8a8e1b6b819d20c070b643c40edc5"}
Jul 10 14:09:31.127650 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.127626    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hcxqk" event={"ID":"f45056ea-8aa6-4200-a036-2591f6364b9a","Type":"ContainerStarted","Data":"80e41db71ba5dd49ecf1ff6ff84f4fadde3280409b14c68d3efde7ada443aa57"}
Jul 10 14:09:31.145571 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.145520    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m8tpr" podStartSLOduration=2.816606224 podStartE2EDuration="4.145502898s" totalImagesPullingTime="1.264289896s" totalInitContainerRuntime="64.606778ms" isStatefulPod=true podCreationTimestamp="2026-07-10 14:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:31.145118943 +0000 UTC m=+42.770844876" watchObservedRunningTime="2026-07-10 14:09:31.145502898 +0000 UTC m=+42.771228828"
Jul 10 14:09:31.172370 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172333    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.172509 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172414    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adad8394-cc75-4339-9898-a25835beef4f-metrics-client-ca\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.172509 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172453    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.172509 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.172680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-tls\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.172680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172579    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxhx5\" (UniqueName: \"kubernetes.io/projected/adad8394-cc75-4339-9898-a25835beef4f-kube-api-access-qxhx5\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.172680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.172680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.172656    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-grpc-tls\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.174881 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.174215    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/adad8394-cc75-4339-9898-a25835beef4f-metrics-client-ca\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.181233 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.181178    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.187324 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.187292    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.187574 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.187550    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.188010 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.187986    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.188559 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.188537    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-grpc-tls\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.191087 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.191047    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxhx5\" (UniqueName: \"kubernetes.io/projected/adad8394-cc75-4339-9898-a25835beef4f-kube-api-access-qxhx5\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.191972 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.191875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/adad8394-cc75-4339-9898-a25835beef4f-secret-thanos-querier-tls\") pod \"thanos-querier-6845b84675-k9755\" (UID: \"adad8394-cc75-4339-9898-a25835beef4f\") " pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.222499 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.222173    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:31.433532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.433497    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6845b84675-k9755"]
Jul 10 14:09:31.438066 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:31.438032    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadad8394_cc75_4339_9898_a25835beef4f.slice/crio-7d78587b0c249a97f62386620098759a3d236119647d82111e4ba2ad69b7b184 WatchSource:0}: Error finding container 7d78587b0c249a97f62386620098759a3d236119647d82111e4ba2ad69b7b184: Status 404 returned error can't find the container with id 7d78587b0c249a97f62386620098759a3d236119647d82111e4ba2ad69b7b184
Jul 10 14:09:31.487063 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.487028    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:31.547603 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:31.547489    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:32.132360 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.132322    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" event={"ID":"adad8394-cc75-4339-9898-a25835beef4f","Type":"ContainerStarted","Data":"7d78587b0c249a97f62386620098759a3d236119647d82111e4ba2ad69b7b184"}
Jul 10 14:09:32.218118 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.218086    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"]
Jul 10 14:09:32.232963 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.232936    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"]
Jul 10 14:09:32.233120 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.233038    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.236338 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.236314    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 10 14:09:32.237781 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.237764    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 10 14:09:32.237868 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.237771    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 10 14:09:32.238170 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.238153    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ursots4cp89p\""
Jul 10 14:09:32.238254 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.238206    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 14:09:32.238295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.238276    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tgb7j\""
Jul 10 14:09:32.285006 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.284973    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lprvd\" (UniqueName: \"kubernetes.io/projected/5b565e78-cded-4cd1-9283-30cf82be7997-kube-api-access-lprvd\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.285157 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.285048    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-secret-metrics-server-client-certs\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.285157 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.285079    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5b565e78-cded-4cd1-9283-30cf82be7997-audit-log\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.285157 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.285136    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5b565e78-cded-4cd1-9283-30cf82be7997-metrics-server-audit-profiles\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.285275 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.285177    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b565e78-cded-4cd1-9283-30cf82be7997-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.285275 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.285220    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-secret-metrics-server-tls\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.285362 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.285295    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-client-ca-bundle\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.385692 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.385612    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lprvd\" (UniqueName: \"kubernetes.io/projected/5b565e78-cded-4cd1-9283-30cf82be7997-kube-api-access-lprvd\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.385692 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.385665    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-secret-metrics-server-client-certs\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.385975 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.385697    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5b565e78-cded-4cd1-9283-30cf82be7997-audit-log\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.385975 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.385750    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5b565e78-cded-4cd1-9283-30cf82be7997-metrics-server-audit-profiles\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.385975 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.385779    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b565e78-cded-4cd1-9283-30cf82be7997-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.385975 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.385805    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-secret-metrics-server-tls\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.385975 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.385831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-client-ca-bundle\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.386226 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.386124    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5b565e78-cded-4cd1-9283-30cf82be7997-audit-log\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.386641 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.386613    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b565e78-cded-4cd1-9283-30cf82be7997-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.386957 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.386934    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5b565e78-cded-4cd1-9283-30cf82be7997-metrics-server-audit-profiles\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.389549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.389522    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-secret-metrics-server-tls\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.389647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.389529    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-client-ca-bundle\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.389647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.389547    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5b565e78-cded-4cd1-9283-30cf82be7997-secret-metrics-server-client-certs\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.395702 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.395680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lprvd\" (UniqueName: \"kubernetes.io/projected/5b565e78-cded-4cd1-9283-30cf82be7997-kube-api-access-lprvd\") pod \"metrics-server-6d4fc76bcd-5m4gs\" (UID: \"5b565e78-cded-4cd1-9283-30cf82be7997\") " pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.543118 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.543082    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:32.601694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.601586    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:32.616882 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.616855    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"]
Jul 10 14:09:32.640126 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.639337    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"]
Jul 10 14:09:32.640126 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.639448    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:32.643841 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.643356    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gsl45\""
Jul 10 14:09:32.643841 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.643637    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 10 14:09:32.666878 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.666844    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:32.688488 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.688454    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tkws\" (UniqueName: \"kubernetes.io/projected/f057f08c-6ce1-448e-a548-0af7467f0739-kube-api-access-8tkws\") pod \"monitoring-plugin-5f68df7f58-qxpqs\" (UID: \"f057f08c-6ce1-448e-a548-0af7467f0739\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:32.688634 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.688503    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f057f08c-6ce1-448e-a548-0af7467f0739-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qxpqs\" (UID: \"f057f08c-6ce1-448e-a548-0af7467f0739\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:32.789385 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.789260    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tkws\" (UniqueName: \"kubernetes.io/projected/f057f08c-6ce1-448e-a548-0af7467f0739-kube-api-access-8tkws\") pod \"monitoring-plugin-5f68df7f58-qxpqs\" (UID: \"f057f08c-6ce1-448e-a548-0af7467f0739\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:32.789385 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.789309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f057f08c-6ce1-448e-a548-0af7467f0739-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qxpqs\" (UID: \"f057f08c-6ce1-448e-a548-0af7467f0739\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:32.789615 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:32.789436    2584 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 10 14:09:32.789615 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:09:32.789504    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f057f08c-6ce1-448e-a548-0af7467f0739-monitoring-plugin-cert podName:f057f08c-6ce1-448e-a548-0af7467f0739 nodeName:}" failed. No retries permitted until 2026-07-10 14:09:33.289486806 +0000 UTC m=+44.915212718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f057f08c-6ce1-448e-a548-0af7467f0739-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-qxpqs" (UID: "f057f08c-6ce1-448e-a548-0af7467f0739") : secret "monitoring-plugin-cert" not found
Jul 10 14:09:32.800288 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.800261    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tkws\" (UniqueName: \"kubernetes.io/projected/f057f08c-6ce1-448e-a548-0af7467f0739-kube-api-access-8tkws\") pod \"monitoring-plugin-5f68df7f58-qxpqs\" (UID: \"f057f08c-6ce1-448e-a548-0af7467f0739\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:32.887854 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.887817    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:32.943276 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:32.943190    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:33.048050 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.048012    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"]
Jul 10 14:09:33.078262 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.078203    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"]
Jul 10 14:09:33.078426 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.078368    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.081979 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.081955    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 10 14:09:33.082327 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.082291    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 10 14:09:33.082327 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.082327    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 10 14:09:33.082466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.082359    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 10 14:09:33.082979 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.082958    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 10 14:09:33.085271 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.083201    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lmjfm\""
Jul 10 14:09:33.095230 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.095194    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 10 14:09:33.134868 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.134833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:33.192901 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.192862    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzjsp\" (UniqueName: \"kubernetes.io/projected/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-kube-api-access-jzjsp\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.193060 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.192918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-metrics-client-ca\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.193060 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.192979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.193060 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.193049    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-federate-client-tls\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.193246 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.193075    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-telemeter-client-tls\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.193246 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.193126    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-serving-certs-ca-bundle\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.193246 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.193171    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.193246 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.193200    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-secret-telemeter-client\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.293620 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293550    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-serving-certs-ca-bundle\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.293620 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293601    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.293855 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293632    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-secret-telemeter-client\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.293855 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293683    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzjsp\" (UniqueName: \"kubernetes.io/projected/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-kube-api-access-jzjsp\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.293855 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-metrics-client-ca\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.293855 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293763    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f057f08c-6ce1-448e-a548-0af7467f0739-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qxpqs\" (UID: \"f057f08c-6ce1-448e-a548-0af7467f0739\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:33.293855 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293788    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.293855 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293854    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-federate-client-tls\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.294164 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.293882    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-telemeter-client-tls\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.295121 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.294372    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-serving-certs-ca-bundle\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.295121 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.295068    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-metrics-client-ca\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.295308 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.295261    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.296761 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.296700    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.296761 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.296730    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f057f08c-6ce1-448e-a548-0af7467f0739-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-qxpqs\" (UID: \"f057f08c-6ce1-448e-a548-0af7467f0739\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:33.296912 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.296799    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-telemeter-client-tls\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.297216 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.297191    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-federate-client-tls\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.297669 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.297645    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-secret-telemeter-client\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.303957 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.303931    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzjsp\" (UniqueName: \"kubernetes.io/projected/7f7669a7-7bfe-4698-b6af-120ce0f7e7d1-kube-api-access-jzjsp\") pod \"telemeter-client-78b8b5d5bd-x8fql\" (UID: \"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1\") " pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.313965 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.313943    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85dc9d66d4-b8b6q"]
Jul 10 14:09:33.380033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.379991    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85dc9d66d4-b8b6q"]
Jul 10 14:09:33.380208 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.380101    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.383898 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.383872    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 14:09:33.384159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.384128    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 14:09:33.384482 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.384463    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 14:09:33.384683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.384664    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 14:09:33.384927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.384909    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 14:09:33.385148 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.385130    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-89tl4\""
Jul 10 14:09:33.385300 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.385290    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 14:09:33.385501 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.385485    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 14:09:33.391111 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.391066    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 14:09:33.393992 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.392681    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"
Jul 10 14:09:33.395483 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.395460    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-service-ca\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.395582 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.395499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkvl2\" (UniqueName: \"kubernetes.io/projected/1bced68a-a31a-492a-8486-7de803446204-kube-api-access-rkvl2\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.395582 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.395538    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-oauth-config\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.395694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.395598    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-trusted-ca-bundle\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.395694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.395625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-serving-cert\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.395694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.395648    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-oauth-serving-cert\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.395854 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.395696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-console-config\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.496225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.496194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkvl2\" (UniqueName: \"kubernetes.io/projected/1bced68a-a31a-492a-8486-7de803446204-kube-api-access-rkvl2\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.496418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.496243    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-oauth-config\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.496418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.496295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-trusted-ca-bundle\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.496418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.496320    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-serving-cert\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.496418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.496355    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-oauth-serving-cert\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.496418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.496400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-console-config\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.496665 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.496466    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-service-ca\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.497283 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.497255    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-service-ca\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.497410 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.497299    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-console-config\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.497410 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.497320    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-trusted-ca-bundle\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.497410 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.497367    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-oauth-serving-cert\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.499205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.499184    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-oauth-config\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.499205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.499209    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-serving-cert\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.506918 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.506880    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkvl2\" (UniqueName: \"kubernetes.io/projected/1bced68a-a31a-492a-8486-7de803446204-kube-api-access-rkvl2\") pod \"console-85dc9d66d4-b8b6q\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") " pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:33.552646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.552555    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:33.696185 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:33.696147    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:34.071186 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.071147    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:34.081041 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.081010    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:09:34.089525 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.089505    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.094001 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.093924    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 14:09:34.094001 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.093939    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 14:09:34.094520 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.094325    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 14:09:34.094520 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.094350    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 14:09:34.094520 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.094458    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 14:09:34.094520 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.094351    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 14:09:34.094793 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.094713    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 14:09:34.094793 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.094740    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3o7koc5e7igfo\""
Jul 10 14:09:34.097947 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.097359    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qlnbg\""
Jul 10 14:09:34.097947 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.097439    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 14:09:34.097947 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.097485    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 14:09:34.097947 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.097583    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.098586    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.101694    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102289    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102329    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102359    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102415    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102464    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-web-config\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-config\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102778    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102821    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102887    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.102922    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103049    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103079    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-config-out\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.103929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103248    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.104794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103298    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.104794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103338    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.104794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103472    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.104794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103647    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs2gr\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-kube-api-access-hs2gr\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.104794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.103759    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.104794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.104048    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 14:09:34.105899 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.105878    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:09:34.106916 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.106892    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 14:09:34.149446 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.149411    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:34.204622 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204592    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.204788 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204634    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.204788 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204758    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.204902 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204788    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.204902 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204820    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.204902 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204853    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-config-out\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.204902 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204880    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205030 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204906    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205030 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204934    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205030 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204964    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205030 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.204993    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs2gr\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-kube-api-access-hs2gr\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205030 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205020    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205228 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205098    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205228 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.205316 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205292    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205459    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205501    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205528    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205533    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205601    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-web-config\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205629    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205661    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-config\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.205818    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.206151    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.206682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.206574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.207265 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.206871    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.207265 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.207027    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.208624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.208598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.208981 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.208957    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.210005 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.209984    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-config\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.210383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.210363    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.211133 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.210800    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-config-out\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.211133 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.211053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.211133 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.211072    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.211133 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.211082    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.211485 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.211463    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-web-config\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.211663 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.211641    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.212834 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.212809    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.212935 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.212907    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.220220 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.220199    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs2gr\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-kube-api-access-hs2gr\") pod \"prometheus-k8s-0\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.403188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.403123    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:34.551831 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.551795    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:34.605415 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:34.605372    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:36.831510 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:36.831466    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:36.932478 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:36.932372    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:37.223846 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:37.223103    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:37.273102 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:37.271031    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"]
Jul 10 14:09:37.336676 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:37.336630    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:37.490369 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:37.490331    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"]
Jul 10 14:09:37.492243 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:37.492220    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf057f08c_6ce1_448e_a548_0af7467f0739.slice/crio-7e6f4465b438ac08446222e84aa45f4c779e49eec022b553675ea36f4ef6c16b WatchSource:0}: Error finding container 7e6f4465b438ac08446222e84aa45f4c779e49eec022b553675ea36f4ef6c16b: Status 404 returned error can't find the container with id 7e6f4465b438ac08446222e84aa45f4c779e49eec022b553675ea36f4ef6c16b
Jul 10 14:09:37.507223 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:37.507128    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:09:37.512347 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:37.512312    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829bb798_d694_4335_9920_e36a5fda0e99.slice/crio-d75b8c5925d3267027569844a1d5da2fe401dcae9364dcd593d1d6e7b127a38a WatchSource:0}: Error finding container d75b8c5925d3267027569844a1d5da2fe401dcae9364dcd593d1d6e7b127a38a: Status 404 returned error can't find the container with id d75b8c5925d3267027569844a1d5da2fe401dcae9364dcd593d1d6e7b127a38a
Jul 10 14:09:37.530153 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:37.530039    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85dc9d66d4-b8b6q"]
Jul 10 14:09:37.533829 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:37.533792    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bced68a_a31a_492a_8486_7de803446204.slice/crio-17a6ebf68e14685112673b3c3e482e7ae25df6908073f408024697397151d5cf WatchSource:0}: Error finding container 17a6ebf68e14685112673b3c3e482e7ae25df6908073f408024697397151d5cf: Status 404 returned error can't find the container with id 17a6ebf68e14685112673b3c3e482e7ae25df6908073f408024697397151d5cf
Jul 10 14:09:37.544182 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:37.544109    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql"]
Jul 10 14:09:37.548933 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:37.548903    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7669a7_7bfe_4698_b6af_120ce0f7e7d1.slice/crio-feeb5a3d02e8b267f68a0d610e07fcccb330db29af2b98eb1176c47181baeb8c WatchSource:0}: Error finding container feeb5a3d02e8b267f68a0d610e07fcccb330db29af2b98eb1176c47181baeb8c: Status 404 returned error can't find the container with id feeb5a3d02e8b267f68a0d610e07fcccb330db29af2b98eb1176c47181baeb8c
Jul 10 14:09:38.161687 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.161650    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kvltq" event={"ID":"cd39c349-df40-4932-92b7-a29e8c11e195","Type":"ContainerStarted","Data":"57f71b8c0fced1d671798e8d799095f31593f1ef56de7aa83a83327b4dc7717e"}
Jul 10 14:09:38.164216 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.164139    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dc9d66d4-b8b6q" event={"ID":"1bced68a-a31a-492a-8486-7de803446204","Type":"ContainerStarted","Data":"17a6ebf68e14685112673b3c3e482e7ae25df6908073f408024697397151d5cf"}
Jul 10 14:09:38.167319 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.167288    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjb47" event={"ID":"75c35095-9063-4aa7-a7c8-4e518a91821b","Type":"ContainerStarted","Data":"73b3d54e80653255d710f6dd71ac0b4cb796f3a151d6440e2ce00ee3cf03f50b"}
Jul 10 14:09:38.168832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.168806    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs" event={"ID":"5b565e78-cded-4cd1-9283-30cf82be7997","Type":"ContainerStarted","Data":"3ead0ce907cb71adc88eed63562418a1ad3b1a06a3d134f9e1fd906a941df5d9"}
Jul 10 14:09:38.171571 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.171545    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hcxqk" event={"ID":"f45056ea-8aa6-4200-a036-2591f6364b9a","Type":"ContainerStarted","Data":"89906f449d9115c325ba0fe20792a4d4eb2fc92f4dd9d7f3aefb2b6e865968ed"}
Jul 10 14:09:38.171684 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.171592    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hcxqk" event={"ID":"f45056ea-8aa6-4200-a036-2591f6364b9a","Type":"ContainerStarted","Data":"8783ab6d6e2623795061604525b2b3e9d2b0b91dc350a65813683c0893005b3b"}
Jul 10 14:09:38.171764 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.171742    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:38.173732 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.173693    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerStarted","Data":"d75b8c5925d3267027569844a1d5da2fe401dcae9364dcd593d1d6e7b127a38a"}
Jul 10 14:09:38.177187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.176998    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" event={"ID":"adad8394-cc75-4339-9898-a25835beef4f","Type":"ContainerStarted","Data":"67111353c1eb071a2680d9afb3e048a81dbb790a3b95da8eb3e6520abba59059"}
Jul 10 14:09:38.177187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.177067    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" event={"ID":"adad8394-cc75-4339-9898-a25835beef4f","Type":"ContainerStarted","Data":"fc8d953c41464441586743420e11569f7379b226222bee68c9696812d70d4730"}
Jul 10 14:09:38.177187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.177107    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" event={"ID":"adad8394-cc75-4339-9898-a25835beef4f","Type":"ContainerStarted","Data":"fce6b152646d10c5a56470c9813bf3f6362b873a463267f8414d8dcc206b75f1"}
Jul 10 14:09:38.178483 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.178418    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs" event={"ID":"f057f08c-6ce1-448e-a548-0af7467f0739","Type":"ContainerStarted","Data":"7e6f4465b438ac08446222e84aa45f4c779e49eec022b553675ea36f4ef6c16b"}
Jul 10 14:09:38.181572 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.181550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql" event={"ID":"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1","Type":"ContainerStarted","Data":"feeb5a3d02e8b267f68a0d610e07fcccb330db29af2b98eb1176c47181baeb8c"}
Jul 10 14:09:38.205031 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:38.204964    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kvltq" podStartSLOduration=2.403927293 podStartE2EDuration="9.204944678s" totalImagesPullingTime="6.801017385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:38.181042327 +0000 UTC m=+49.806768257" watchObservedRunningTime="2026-07-10 14:09:38.204944678 +0000 UTC m=+49.830670611"
Jul 10 14:09:39.106526 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:39.106486    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:39.212075 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:39.212039    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:39.954701 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:39.954617    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:40.053246 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:40.053211    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:43.209376 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.209340    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dc9d66d4-b8b6q" event={"ID":"1bced68a-a31a-492a-8486-7de803446204","Type":"ContainerStarted","Data":"9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b"}
Jul 10 14:09:43.211337 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.211308    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjb47" event={"ID":"75c35095-9063-4aa7-a7c8-4e518a91821b","Type":"ContainerStarted","Data":"98dedbce7a1b2bc9531bfff9e845d6a6e55bbd87366fe90b4e580c88b0db1b25"}
Jul 10 14:09:43.212763 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.212733    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs" event={"ID":"5b565e78-cded-4cd1-9283-30cf82be7997","Type":"ContainerStarted","Data":"b0da2cdd2eb839e6b7d1f4a98a45aa8411cc2bc394d4acc0d8c09d15ef38049c"}
Jul 10 14:09:43.214596 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.214575    2584 generic.go:356] "Generic (PLEG): container finished" podID="829bb798-d694-4335-9920-e36a5fda0e99" containerID="ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df" exitCode=0
Jul 10 14:09:43.214691 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.214648    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df"}
Jul 10 14:09:43.217963 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.217929    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" event={"ID":"adad8394-cc75-4339-9898-a25835beef4f","Type":"ContainerStarted","Data":"7355422fcf283ef137e1ef5de77b2aa73ebdf8054b22f4bda41b51c421944c82"}
Jul 10 14:09:43.218063 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.217993    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" event={"ID":"adad8394-cc75-4339-9898-a25835beef4f","Type":"ContainerStarted","Data":"3d27123a06f5ce9f11842fb2240c91187223151c01bc9a433ab2cc291a3b6472"}
Jul 10 14:09:43.218063 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.218037    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" event={"ID":"adad8394-cc75-4339-9898-a25835beef4f","Type":"ContainerStarted","Data":"459c2fbebe386ff1bb6fd087737956dadbad5fb6bde19b54120164a52bcd4ea0"}
Jul 10 14:09:43.218170 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.218079    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:43.219822 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.219788    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs" event={"ID":"f057f08c-6ce1-448e-a548-0af7467f0739","Type":"ContainerStarted","Data":"5eceaba5cc97e36635db3d9853c1e39398dadab4ec1f9e45c08d8ddc07ff3b12"}
Jul 10 14:09:43.219995 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.219894    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:43.222228 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.222192    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql" event={"ID":"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1","Type":"ContainerStarted","Data":"3a2578913806d582df59e1f7129a012f4e0101ad1edf5f61afcce8141bc0b478"}
Jul 10 14:09:43.222434 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.222411    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql" event={"ID":"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1","Type":"ContainerStarted","Data":"572faf852c38d52c94d3197979d030b37b10711980f0bd38f0628520187253c2"}
Jul 10 14:09:43.222582 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.222469    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql" event={"ID":"7f7669a7-7bfe-4698-b6af-120ce0f7e7d1","Type":"ContainerStarted","Data":"bc09ebc06407d65d1927941db7ee77dd913a64d4e0a3fd72a9e429cda21b02af"}
Jul 10 14:09:43.224534 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.224511    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6845b84675-k9755"
Jul 10 14:09:43.225856 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.225837    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs"
Jul 10 14:09:43.229542 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.229489    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85dc9d66d4-b8b6q" podStartSLOduration=5.342713869 podStartE2EDuration="10.229473307s" totalImagesPullingTime="4.886759438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:43.228332848 +0000 UTC m=+54.854058782" watchObservedRunningTime="2026-07-10 14:09:43.229473307 +0000 UTC m=+54.855199216"
Jul 10 14:09:43.230285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.230248    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hcxqk" podStartSLOduration=7.423621261 podStartE2EDuration="14.230231569s" totalImagesPullingTime="6.806610308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:38.204663605 +0000 UTC m=+49.830389537" watchObservedRunningTime="2026-07-10 14:09:43.230231569 +0000 UTC m=+54.855957502"
Jul 10 14:09:43.254942 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.254885    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6845b84675-k9755" podStartSLOduration=2.8192715489999998 podStartE2EDuration="13.254865543s" totalImagesPullingTime="10.435593994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:43.254313972 +0000 UTC m=+54.880039904" watchObservedRunningTime="2026-07-10 14:09:43.254865543 +0000 UTC m=+54.880591476"
Jul 10 14:09:43.270635 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.270601    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:43.274917 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.274867    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs" podStartSLOduration=6.137074689 podStartE2EDuration="11.274847404s" totalImagesPullingTime="5.137772715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:43.274402169 +0000 UTC m=+54.900128100" watchObservedRunningTime="2026-07-10 14:09:43.274847404 +0000 UTC m=+54.900573336"
Jul 10 14:09:43.295996 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.295945    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-qxpqs" podStartSLOduration=6.370768768 podStartE2EDuration="11.295930233s" totalImagesPullingTime="4.925161465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:43.295675452 +0000 UTC m=+54.921401384" watchObservedRunningTime="2026-07-10 14:09:43.295930233 +0000 UTC m=+54.921656143"
Jul 10 14:09:43.319590 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.319537    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78b8b5d5bd-x8fql" podStartSLOduration=5.446838635 podStartE2EDuration="10.319519204s" totalImagesPullingTime="4.872680569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:43.318079974 +0000 UTC m=+54.943805909" watchObservedRunningTime="2026-07-10 14:09:43.319519204 +0000 UTC m=+54.945245135"
Jul 10 14:09:43.349896 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.349864    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:43.696609 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.696573    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:43.696803 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.696670    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:43.702021 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:43.701989    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:44.070959 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:44.070924    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85dc9d66d4-b8b6q"]
Jul 10 14:09:44.232380 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:44.232348    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:09:44.271371 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:44.271315    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:44.867242 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:44.867201    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:44.969230 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:44.969199    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:45.087176 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:45.087123    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:45.187773 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:45.187674    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:45.259989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:45.259955    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:46.155391 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:46.155354    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:46.234579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:46.234541    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:46.237518 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:46.237490    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjb47" event={"ID":"75c35095-9063-4aa7-a7c8-4e518a91821b","Type":"ContainerStarted","Data":"2a61e1c54089e85bb047738d74a007f07246c6d87acfd7de1efc4d6aeec15437"}
Jul 10 14:09:46.264853 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:46.264801    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fjb47" podStartSLOduration=4.901554615 podStartE2EDuration="17.264787719s" totalImagesPullingTime="12.363233104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:09:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:46.263820571 +0000 UTC m=+57.889546506" watchObservedRunningTime="2026-07-10 14:09:46.264787719 +0000 UTC m=+57.890513693"
Jul 10 14:09:47.253433 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:47.253076    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerStarted","Data":"5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614"}
Jul 10 14:09:47.253433 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:47.253189    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerStarted","Data":"63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d"}
Jul 10 14:09:47.253433 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:47.253236    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerStarted","Data":"abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa"}
Jul 10 14:09:48.189974 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:48.189937    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hcxqk"
Jul 10 14:09:48.260581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:48.260550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerStarted","Data":"2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003"}
Jul 10 14:09:48.260783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:48.260617    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerStarted","Data":"6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0"}
Jul 10 14:09:48.260783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:48.260642    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerStarted","Data":"a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00"}
Jul 10 14:09:48.291472 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:48.291404    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.859098857 podStartE2EDuration="14.291383942s" totalImagesPullingTime="7.402546403s" totalInitContainerRuntime="29.738682ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:09:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:48.289942628 +0000 UTC m=+59.915668560" watchObservedRunningTime="2026-07-10 14:09:48.291383942 +0000 UTC m=+59.917109873"
Jul 10 14:09:49.403825 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:49.403786    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:09:51.023602 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:51.023551    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:51.121729 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:51.121681    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:52.543426 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:52.543391    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:52.543831 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:52.543457    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:09:54.027379 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.027342    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:54.106687 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.106654    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:54.501476 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.501435    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:54.504609 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.504593    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 14:09:54.514445 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.514426    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259a5382-782c-4cba-8bee-effd8d3b49fc-original-pull-secret\") pod \"global-pull-secret-syncer-cm2np\" (UID: \"259a5382-782c-4cba-8bee-effd8d3b49fc\") " pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:54.602598 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.602552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:54.605425 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.605409    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 14:09:54.615796 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.615770    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b40128e-bfe7-46ea-baa5-edd38a752eb7-metrics-certs\") pod \"network-metrics-daemon-qmqv2\" (UID: \"2b40128e-bfe7-46ea-baa5-edd38a752eb7\") " pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:54.630823 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.630799    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cm2np"
Jul 10 14:09:54.651767 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.651746    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fptgw\""
Jul 10 14:09:54.659079 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.659061    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qmqv2"
Jul 10 14:09:54.703330 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.703295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:54.706447 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.706390    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 14:09:54.717475 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.717450    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 14:09:54.727842 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.727784    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbpfc\" (UniqueName: \"kubernetes.io/projected/cba8fa69-2989-4577-b3dd-bdccc1e02ce0-kube-api-access-jbpfc\") pod \"network-check-target-5trbg\" (UID: \"cba8fa69-2989-4577-b3dd-bdccc1e02ce0\") " pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:54.794663 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.794630    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cm2np"]
Jul 10 14:09:54.796688 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:54.796664    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259a5382_782c_4cba_8bee_effd8d3b49fc.slice/crio-f02ff98eeebaa301d40dce05db719974d8ed74b76b1f1959d8a3d60b542d5e23 WatchSource:0}: Error finding container f02ff98eeebaa301d40dce05db719974d8ed74b76b1f1959d8a3d60b542d5e23: Status 404 returned error can't find the container with id f02ff98eeebaa301d40dce05db719974d8ed74b76b1f1959d8a3d60b542d5e23
Jul 10 14:09:54.823634 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.823611    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qmqv2"]
Jul 10 14:09:54.825858 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:54.825833    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b40128e_bfe7_46ea_baa5_edd38a752eb7.slice/crio-975ce01b0630318e2f684ed1e2012e359d0457cdd037b28fe0d754d82311a15c WatchSource:0}: Error finding container 975ce01b0630318e2f684ed1e2012e359d0457cdd037b28fe0d754d82311a15c: Status 404 returned error can't find the container with id 975ce01b0630318e2f684ed1e2012e359d0457cdd037b28fe0d754d82311a15c
Jul 10 14:09:54.943567 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.943540    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7ddd8\""
Jul 10 14:09:54.951081 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:54.951064    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:09:55.072481 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.072439    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5trbg"]
Jul 10 14:09:55.074890 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:09:55.074469    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcba8fa69_2989_4577_b3dd_bdccc1e02ce0.slice/crio-9ce9a5c334f573083f9057f7ea56fb0cc64477bda466ba22b7f9e0d185d494a4 WatchSource:0}: Error finding container 9ce9a5c334f573083f9057f7ea56fb0cc64477bda466ba22b7f9e0d185d494a4: Status 404 returned error can't find the container with id 9ce9a5c334f573083f9057f7ea56fb0cc64477bda466ba22b7f9e0d185d494a4
Jul 10 14:09:55.085571 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.085550    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hz6t"
Jul 10 14:09:55.199877 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.199836    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:55.290388 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.290352    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qmqv2" event={"ID":"2b40128e-bfe7-46ea-baa5-edd38a752eb7","Type":"ContainerStarted","Data":"975ce01b0630318e2f684ed1e2012e359d0457cdd037b28fe0d754d82311a15c"}
Jul 10 14:09:55.291879 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.291838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5trbg" event={"ID":"cba8fa69-2989-4577-b3dd-bdccc1e02ce0","Type":"ContainerStarted","Data":"9ce9a5c334f573083f9057f7ea56fb0cc64477bda466ba22b7f9e0d185d494a4"}
Jul 10 14:09:55.293386 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.293351    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cm2np" event={"ID":"259a5382-782c-4cba-8bee-effd8d3b49fc","Type":"ContainerStarted","Data":"f02ff98eeebaa301d40dce05db719974d8ed74b76b1f1959d8a3d60b542d5e23"}
Jul 10 14:09:55.300793 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.300768    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:55.362084 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:55.362008    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:56.300369 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:56.300284    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qmqv2" event={"ID":"2b40128e-bfe7-46ea-baa5-edd38a752eb7","Type":"ContainerStarted","Data":"d833896ba8dd5568b20d96c0edde0a8c1ab636b14c59f2c38c1453bd7a069bfe"}
Jul 10 14:09:56.300369 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:56.300353    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qmqv2" event={"ID":"2b40128e-bfe7-46ea-baa5-edd38a752eb7","Type":"ContainerStarted","Data":"c25f6e28cfcc5a49f8ff89f66d734071d16bb268f673ec605e8f983a94c07b23"}
Jul 10 14:09:56.321171 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:56.321117    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qmqv2" podStartSLOduration=66.176901253 podStartE2EDuration="1m7.32109438s" totalImagesPullingTime="1.144193127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:09:56.319929387 +0000 UTC m=+67.945655319" watchObservedRunningTime="2026-07-10 14:09:56.32109438 +0000 UTC m=+67.946820311"
Jul 10 14:09:56.429113 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:56.429082    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:09:56.489931 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:09:56.489901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:00.036011 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:00.035970    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:00.094556 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:00.094480    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:00.317885 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:00.317851    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cm2np" event={"ID":"259a5382-782c-4cba-8bee-effd8d3b49fc","Type":"ContainerStarted","Data":"0e577bb727756670399248fd0d6eacd255325aa7dcb75ce2eeebd697a1c6a7c5"}
Jul 10 14:10:00.319417 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:00.319392    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5trbg" event={"ID":"cba8fa69-2989-4577-b3dd-bdccc1e02ce0","Type":"ContainerStarted","Data":"132dc02db797daf62810f1b2a2f8624bbfdbb1b46d1d9c2539a19433f9ead84e"}
Jul 10 14:10:00.319544 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:00.319518    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:10:00.333791 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:00.333740    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cm2np" podStartSLOduration=66.284389971 podStartE2EDuration="1m11.333702599s" totalImagesPullingTime="5.049312628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:10:00.333260607 +0000 UTC m=+71.958986539" watchObservedRunningTime="2026-07-10 14:10:00.333702599 +0000 UTC m=+71.959428532"
Jul 10 14:10:00.348904 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:00.348831    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5trbg" podStartSLOduration=66.580484263 podStartE2EDuration="1m11.348818595s" totalImagesPullingTime="4.768334332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:10:00.347600798 +0000 UTC m=+71.973326728" watchObservedRunningTime="2026-07-10 14:10:00.348818595 +0000 UTC m=+71.974544526"
Jul 10 14:10:08.903470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:08.903439    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:08.969440 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:08.969412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:09.333673 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:09.333616    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:09.399665 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:09.399631    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:10.259488 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.259337    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85dc9d66d4-b8b6q" podUID="1bced68a-a31a-492a-8486-7de803446204" containerName="console" containerID="cri-o://9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b" gracePeriod=15
Jul 10 14:10:10.493161 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.493137    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:10:10.548833 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.548752    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-serving-cert\") pod \"1bced68a-a31a-492a-8486-7de803446204\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") "
Jul 10 14:10:10.548833 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.548796    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-oauth-config\") pod \"1bced68a-a31a-492a-8486-7de803446204\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") "
Jul 10 14:10:10.548833 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.548817    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-service-ca\") pod \"1bced68a-a31a-492a-8486-7de803446204\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") "
Jul 10 14:10:10.549103 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.548840    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-oauth-serving-cert\") pod \"1bced68a-a31a-492a-8486-7de803446204\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") "
Jul 10 14:10:10.549103 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549016    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-console-config\") pod \"1bced68a-a31a-492a-8486-7de803446204\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") "
Jul 10 14:10:10.549103 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549079    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkvl2\" (UniqueName: \"kubernetes.io/projected/1bced68a-a31a-492a-8486-7de803446204-kube-api-access-rkvl2\") pod \"1bced68a-a31a-492a-8486-7de803446204\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") "
Jul 10 14:10:10.549260 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549114    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-trusted-ca-bundle\") pod \"1bced68a-a31a-492a-8486-7de803446204\" (UID: \"1bced68a-a31a-492a-8486-7de803446204\") "
Jul 10 14:10:10.549260 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549221    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1bced68a-a31a-492a-8486-7de803446204" (UID: "1bced68a-a31a-492a-8486-7de803446204"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:10.549260 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549216    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-service-ca" (OuterVolumeSpecName: "service-ca") pod "1bced68a-a31a-492a-8486-7de803446204" (UID: "1bced68a-a31a-492a-8486-7de803446204"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:10.549400 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549352    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:10.549400 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549370    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-oauth-serving-cert\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:10.549568 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549486    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-console-config" (OuterVolumeSpecName: "console-config") pod "1bced68a-a31a-492a-8486-7de803446204" (UID: "1bced68a-a31a-492a-8486-7de803446204"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:10.549680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.549650    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1bced68a-a31a-492a-8486-7de803446204" (UID: "1bced68a-a31a-492a-8486-7de803446204"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:10.551543 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.551515    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bced68a-a31a-492a-8486-7de803446204-kube-api-access-rkvl2" (OuterVolumeSpecName: "kube-api-access-rkvl2") pod "1bced68a-a31a-492a-8486-7de803446204" (UID: "1bced68a-a31a-492a-8486-7de803446204"). InnerVolumeSpecName "kube-api-access-rkvl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:10:10.551659 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.551557    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1bced68a-a31a-492a-8486-7de803446204" (UID: "1bced68a-a31a-492a-8486-7de803446204"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:10.551659 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.551632    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1bced68a-a31a-492a-8486-7de803446204" (UID: "1bced68a-a31a-492a-8486-7de803446204"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:10.649766 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.649734    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-serving-cert\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:10.649766 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.649761    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1bced68a-a31a-492a-8486-7de803446204-console-oauth-config\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:10.649766 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.649772    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-console-config\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:10.649976 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.649780    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkvl2\" (UniqueName: \"kubernetes.io/projected/1bced68a-a31a-492a-8486-7de803446204-kube-api-access-rkvl2\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:10.649976 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:10.649790    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bced68a-a31a-492a-8486-7de803446204-trusted-ca-bundle\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:11.359373 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.359339    2584 generic.go:356] "Generic (PLEG): container finished" podID="1bced68a-a31a-492a-8486-7de803446204" containerID="9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b" exitCode=0
Jul 10 14:10:11.359795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.359391    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85dc9d66d4-b8b6q"
Jul 10 14:10:11.359795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.359401    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dc9d66d4-b8b6q" event={"ID":"1bced68a-a31a-492a-8486-7de803446204","Type":"ContainerDied","Data":"9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b"}
Jul 10 14:10:11.359795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.359438    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85dc9d66d4-b8b6q" event={"ID":"1bced68a-a31a-492a-8486-7de803446204","Type":"ContainerDied","Data":"17a6ebf68e14685112673b3c3e482e7ae25df6908073f408024697397151d5cf"}
Jul 10 14:10:11.359795 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.359457    2584 scope.go:117] "RemoveContainer" containerID="9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b"
Jul 10 14:10:11.368172 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.368152    2584 scope.go:117] "RemoveContainer" containerID="9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b"
Jul 10 14:10:11.368426 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:11.368404    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b\": container with ID starting with 9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b not found: ID does not exist" containerID="9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b"
Jul 10 14:10:11.368484 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.368434    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b"} err="failed to get container status \"9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b\": rpc error: code = NotFound desc = could not find container \"9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b\": container with ID starting with 9e4f246bb0dc25023e5e8f629fa0540ba85a63fbb0ac4d2ee172fdfa2f596c2b not found: ID does not exist"
Jul 10 14:10:11.378232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.378211    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85dc9d66d4-b8b6q"]
Jul 10 14:10:11.383671 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:11.383649    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85dc9d66d4-b8b6q"]
Jul 10 14:10:12.549014 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:12.548983    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:10:12.552885 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:12.552862    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d4fc76bcd-5m4gs"
Jul 10 14:10:12.919646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:12.919613    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bced68a-a31a-492a-8486-7de803446204" path="/var/lib/kubelet/pods/1bced68a-a31a-492a-8486-7de803446204/volumes"
Jul 10 14:10:15.478926 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:15.478891    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:15.526837 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:15.526804    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:16.393983 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:16.393960    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:18.291301 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:18.291262    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:18.393018 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:18.392983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:18.493673 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:18.493643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:23.922299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:23.922265    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:23.980356 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:23.980323    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:30.388182 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:30.388151    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:30.488549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:30.488509    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:31.325453 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:31.325422    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5trbg"
Jul 10 14:10:31.871934 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:31.871901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:32.452894 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.452864    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:10:32.453224 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.453192    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="prometheus" containerID="cri-o://abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" gracePeriod=600
Jul 10 14:10:32.453343 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.453252    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="config-reloader" containerID="cri-o://63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d" gracePeriod=600
Jul 10 14:10:32.453343 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.453201    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy" containerID="cri-o://6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0" gracePeriod=600
Jul 10 14:10:32.453467 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.453231    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-web" containerID="cri-o://a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00" gracePeriod=600
Jul 10 14:10:32.453467 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.453229    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="thanos-sidecar" containerID="cri-o://5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614" gracePeriod=600
Jul 10 14:10:32.453467 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.453252    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003" gracePeriod=600
Jul 10 14:10:32.757274 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.757245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:32.813627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:32.813424    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:33.451977 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:33.451789    2584 generic.go:356] "Generic (PLEG): container finished" podID="829bb798-d694-4335-9920-e36a5fda0e99" containerID="5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614" exitCode=0
Jul 10 14:10:33.451977 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:33.451815    2584 generic.go:356] "Generic (PLEG): container finished" podID="829bb798-d694-4335-9920-e36a5fda0e99" containerID="63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d" exitCode=0
Jul 10 14:10:33.451977 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:33.451838    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614"}
Jul 10 14:10:33.451977 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:33.451924    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d"}
Jul 10 14:10:34.461154 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:34.461123    2584 generic.go:356] "Generic (PLEG): container finished" podID="829bb798-d694-4335-9920-e36a5fda0e99" containerID="2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003" exitCode=0
Jul 10 14:10:34.461154 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:34.461147    2584 generic.go:356] "Generic (PLEG): container finished" podID="829bb798-d694-4335-9920-e36a5fda0e99" containerID="6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0" exitCode=0
Jul 10 14:10:34.461154 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:34.461153    2584 generic.go:356] "Generic (PLEG): container finished" podID="829bb798-d694-4335-9920-e36a5fda0e99" containerID="a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00" exitCode=0
Jul 10 14:10:34.461730 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:34.461200    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003"}
Jul 10 14:10:34.461730 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:34.461295    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0"}
Jul 10 14:10:34.461730 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:34.461340    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00"}
Jul 10 14:10:36.380492 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:36.380451    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 10 14:10:36.381452 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:36.381422    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 10 14:10:36.382369 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:36.382331    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 10 14:10:36.382471 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:36.382385    2584 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="prometheus" probeResult="unknown"
Jul 10 14:10:39.045629 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:39.045586    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:39.145211 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:39.145180    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:39.247194 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:39.247160    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:41.380934 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:41.380884    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 10 14:10:41.381916 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:41.381873    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 10 14:10:41.382781 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:41.382753    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 10 14:10:41.382870 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:41.382793    2584 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="prometheus" probeResult="unknown"
Jul 10 14:10:41.892973 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.892947    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:41.999045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.998957    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs2gr\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-kube-api-access-hs2gr\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999001    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-config-out\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999025    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-kubelet-serving-ca-bundle\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999045    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-trusted-ca-bundle\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999081    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-2\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999100    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-1\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999139    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999165    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-metrics-client-certs\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999194    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-db\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999227    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-web-config\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999256    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999282    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-config\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999314    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-grpc-tls\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999375    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-tls\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:41.999880 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999422    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:41.999880 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999527    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:41.999880 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999835    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:42.000043 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:41.999895    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000216    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-thanos-prometheus-http-client-file\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000266    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-metrics-client-ca\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000295    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-0\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000321    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-kube-rbac-proxy\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000356    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-tls-assets\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000391    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-serving-certs-ca-bundle\") pod \"829bb798-d694-4335-9920-e36a5fda0e99\" (UID: \"829bb798-d694-4335-9920-e36a5fda0e99\") "
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000525    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000663    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000685    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000700    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.000799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000731    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.001408 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.000832    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-db\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.002400 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.002017    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:42.002400 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.002134    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:42.002400 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.002363    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:10:42.003177 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.003150    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-kube-api-access-hs2gr" (OuterVolumeSpecName: "kube-api-access-hs2gr") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "kube-api-access-hs2gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:10:42.003653 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.003613    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-config-out" (OuterVolumeSpecName: "config-out") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:10:42.004039 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.003934    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.004420 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.004394    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.004648 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.004611    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.004785 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.004761    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.004852 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.004665    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.004913 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.004874    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-config" (OuterVolumeSpecName: "config") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.004978 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.004955    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.005025 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.004983    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:10:42.005389 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.005368    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.013403 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.013382    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-web-config" (OuterVolumeSpecName: "web-config") pod "829bb798-d694-4335-9920-e36a5fda0e99" (UID: "829bb798-d694-4335-9920-e36a5fda0e99"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:10:42.101944 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.101916    2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/829bb798-d694-4335-9920-e36a5fda0e99-config-out\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.101944 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.101942    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.101956    2584 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-metrics-client-certs\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.101986    2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-web-config\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.101996    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102005    2584 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-config\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102013    2584 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-grpc-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102022    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102032    2584 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102045    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-metrics-client-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102053    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102061    2584 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/829bb798-d694-4335-9920-e36a5fda0e99-secret-kube-rbac-proxy\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102069    2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-tls-assets\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102076    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/829bb798-d694-4335-9920-e36a5fda0e99-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.102128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.102084    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs2gr\" (UniqueName: \"kubernetes.io/projected/829bb798-d694-4335-9920-e36a5fda0e99-kube-api-access-hs2gr\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:10:42.378999 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.378966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:42.479017 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.478983    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:42.493465 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.493437    2584 generic.go:356] "Generic (PLEG): container finished" podID="829bb798-d694-4335-9920-e36a5fda0e99" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa" exitCode=0
Jul 10 14:10:42.493580 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.493498    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa"}
Jul 10 14:10:42.493580 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.493525    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.493580 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.493568    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"829bb798-d694-4335-9920-e36a5fda0e99","Type":"ContainerDied","Data":"d75b8c5925d3267027569844a1d5da2fe401dcae9364dcd593d1d6e7b127a38a"}
Jul 10 14:10:42.493675 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.493603    2584 scope.go:117] "RemoveContainer" containerID="2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003"
Jul 10 14:10:42.507088 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.507069    2584 scope.go:117] "RemoveContainer" containerID="6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0"
Jul 10 14:10:42.513695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.513678    2584 scope.go:117] "RemoveContainer" containerID="a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00"
Jul 10 14:10:42.519967 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.519944    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:10:42.520815 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.520799    2584 scope.go:117] "RemoveContainer" containerID="5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614"
Jul 10 14:10:42.525267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.525249    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:10:42.528010 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.527985    2584 scope.go:117] "RemoveContainer" containerID="63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d"
Jul 10 14:10:42.536105 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.536089    2584 scope.go:117] "RemoveContainer" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa"
Jul 10 14:10:42.542891 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.542847    2584 scope.go:117] "RemoveContainer" containerID="ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df"
Jul 10 14:10:42.549422 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.549406    2584 scope.go:117] "RemoveContainer" containerID="2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003"
Jul 10 14:10:42.549690 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:42.549663    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003\": container with ID starting with 2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003 not found: ID does not exist" containerID="2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003"
Jul 10 14:10:42.549776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.549701    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003"} err="failed to get container status \"2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003\": rpc error: code = NotFound desc = could not find container \"2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003\": container with ID starting with 2e03e1ed9b7052f67274f514344af92be82d230ddc9c1c656fb2e78afaf26003 not found: ID does not exist"
Jul 10 14:10:42.549776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.549733    2584 scope.go:117] "RemoveContainer" containerID="6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0"
Jul 10 14:10:42.550007 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:42.549990    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0\": container with ID starting with 6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0 not found: ID does not exist" containerID="6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0"
Jul 10 14:10:42.550052 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550013    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0"} err="failed to get container status \"6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0\": rpc error: code = NotFound desc = could not find container \"6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0\": container with ID starting with 6692e615a4e798f2e063f1a9a970e17794aead89acbfae9bfcf54b8feb6814b0 not found: ID does not exist"
Jul 10 14:10:42.550052 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550028    2584 scope.go:117] "RemoveContainer" containerID="a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00"
Jul 10 14:10:42.550256 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:42.550240    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00\": container with ID starting with a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00 not found: ID does not exist" containerID="a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00"
Jul 10 14:10:42.550299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550261    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00"} err="failed to get container status \"a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00\": rpc error: code = NotFound desc = could not find container \"a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00\": container with ID starting with a3c2fe1be2ad91512af8f6dc8078aaafc249abfb83bd0152aa169f408eba2a00 not found: ID does not exist"
Jul 10 14:10:42.550299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550275    2584 scope.go:117] "RemoveContainer" containerID="5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614"
Jul 10 14:10:42.550515 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:42.550501    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614\": container with ID starting with 5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614 not found: ID does not exist" containerID="5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614"
Jul 10 14:10:42.550553 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550518    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614"} err="failed to get container status \"5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614\": rpc error: code = NotFound desc = could not find container \"5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614\": container with ID starting with 5fa80a1dd2acad16d5a319a72e0a918e3dde453b095f8391d2a70483ac562614 not found: ID does not exist"
Jul 10 14:10:42.550553 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550533    2584 scope.go:117] "RemoveContainer" containerID="63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d"
Jul 10 14:10:42.550757 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:42.550741    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d\": container with ID starting with 63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d not found: ID does not exist" containerID="63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d"
Jul 10 14:10:42.550816 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550761    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d"} err="failed to get container status \"63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d\": rpc error: code = NotFound desc = could not find container \"63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d\": container with ID starting with 63a26fa4632f308df3087f699120a459caa631fb2daa3a50173c5cca26e3e30d not found: ID does not exist"
Jul 10 14:10:42.550816 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.550775    2584 scope.go:117] "RemoveContainer" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa"
Jul 10 14:10:42.551046 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:42.551026    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa\": container with ID starting with abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa not found: ID does not exist" containerID="abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa"
Jul 10 14:10:42.551090 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.551051    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa"} err="failed to get container status \"abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa\": rpc error: code = NotFound desc = could not find container \"abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa\": container with ID starting with abae32bac0d440d738a69b7cb5b173d73cf4613c9f5897cce8489bcb1343bfaa not found: ID does not exist"
Jul 10 14:10:42.551090 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.551065    2584 scope.go:117] "RemoveContainer" containerID="ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df"
Jul 10 14:10:42.551283 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:10:42.551266    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df\": container with ID starting with ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df not found: ID does not exist" containerID="ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df"
Jul 10 14:10:42.551320 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.551289    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df"} err="failed to get container status \"ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df\": rpc error: code = NotFound desc = could not find container \"ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df\": container with ID starting with ea5efb307307b426bfe38552da215b76c52a9ccfad734d1015599889ce7f74df not found: ID does not exist"
Jul 10 14:10:42.581629 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581597    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:10:42.581893 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581880    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-web"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581894    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-web"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581903    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bced68a-a31a-492a-8486-7de803446204" containerName="console"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581909    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bced68a-a31a-492a-8486-7de803446204" containerName="console"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581916    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="thanos-sidecar"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581921    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="thanos-sidecar"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581929    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="config-reloader"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581934    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="config-reloader"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581940    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy"
Jul 10 14:10:42.581941 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581945    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581952    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="init-config-reloader"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581958    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="init-config-reloader"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581963    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="prometheus"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581968    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="prometheus"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581974    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-thanos"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.581979    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-thanos"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.582022    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="config-reloader"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.582029    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bced68a-a31a-492a-8486-7de803446204" containerName="console"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.582036    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-thanos"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.582042    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="prometheus"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.582047    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy-web"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.582052    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="kube-rbac-proxy"
Jul 10 14:10:42.582225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.582058    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="829bb798-d694-4335-9920-e36a5fda0e99" containerName="thanos-sidecar"
Jul 10 14:10:42.585714 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.585698    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.590981 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.590961    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 14:10:42.591254 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.591233    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 14:10:42.592253 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.592233    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 14:10:42.592338 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.592252    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 14:10:42.592773 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.592756    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3o7koc5e7igfo\""
Jul 10 14:10:42.592847 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.592790    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 14:10:42.592847 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.592818    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 14:10:42.592966 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.592876    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qlnbg\""
Jul 10 14:10:42.593058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.593044    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 14:10:42.593119 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.593091    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 14:10:42.593170 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.593143    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 14:10:42.593329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.593303    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 14:10:42.593435 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.593421    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 14:10:42.598274 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.598259    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 14:10:42.607264 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.607250    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 14:10:42.612994 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.612980    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 14:10:42.625306 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.625286    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:42.683660 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.683593    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:10:42.696582 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.696559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:10:42.704708 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704686    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.704838 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704737    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.704838 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704768    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.704838 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704793    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8998f81-303c-420a-88e3-a5e7031d074f-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.704838 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704819    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705003 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705003 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704902    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705003 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704921    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705003 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.704956    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-config\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705203 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhh96\" (UniqueName: \"kubernetes.io/projected/f8998f81-303c-420a-88e3-a5e7031d074f-kube-api-access-dhh96\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705203 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705065    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705203 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705098    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705203 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705160    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705203 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705211    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705267    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705296    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8998f81-303c-420a-88e3-a5e7031d074f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705312    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.705383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.705358    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806152 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806115    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806152 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806358 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8998f81-303c-420a-88e3-a5e7031d074f-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806358 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806195    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806358 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806358 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806358 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806333    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806605 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806371    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-config\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806605 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806422    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhh96\" (UniqueName: \"kubernetes.io/projected/f8998f81-303c-420a-88e3-a5e7031d074f-kube-api-access-dhh96\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806605 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806605 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806567    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806866 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806837    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806875    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.806927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806899    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806932    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806958    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.806993    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8998f81-303c-420a-88e3-a5e7031d074f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807265 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807055    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807265 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807090    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807265 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807128    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807265 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807158    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807674 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807636    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807792 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807700    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.807792 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.807163    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.809285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.809259    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.809384 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.809348    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.810283 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.809934    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-config\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.810283 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.810109    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8998f81-303c-420a-88e3-a5e7031d074f-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.810283 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.810232    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.810535 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.810512    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8998f81-303c-420a-88e3-a5e7031d074f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.811295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.811253    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.811394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.811347    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.811756 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.811738    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8998f81-303c-420a-88e3-a5e7031d074f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.811834 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.811811    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.812245 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.812223    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.812441 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.812423    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.812592 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.812576    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8998f81-303c-420a-88e3-a5e7031d074f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.815202 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.815180    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhh96\" (UniqueName: \"kubernetes.io/projected/f8998f81-303c-420a-88e3-a5e7031d074f-kube-api-access-dhh96\") pod \"prometheus-k8s-0\" (UID: \"f8998f81-303c-420a-88e3-a5e7031d074f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.895317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.895272    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:10:42.919532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:42.919503    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="829bb798-d694-4335-9920-e36a5fda0e99" path="/var/lib/kubelet/pods/829bb798-d694-4335-9920-e36a5fda0e99/volumes"
Jul 10 14:10:43.032889 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:43.032856    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 14:10:43.036383 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:10:43.036352    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8998f81_303c_420a_88e3_a5e7031d074f.slice/crio-9d6703cc4b6c3d2c72f8a209c23399c6d1e92dfe6c92afd7b0ec97216dfee3f0 WatchSource:0}: Error finding container 9d6703cc4b6c3d2c72f8a209c23399c6d1e92dfe6c92afd7b0ec97216dfee3f0: Status 404 returned error can't find the container with id 9d6703cc4b6c3d2c72f8a209c23399c6d1e92dfe6c92afd7b0ec97216dfee3f0
Jul 10 14:10:43.500017 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:43.499979    2584 generic.go:356] "Generic (PLEG): container finished" podID="f8998f81-303c-420a-88e3-a5e7031d074f" containerID="73f87a80deb7cb3b70066a5c87f5cf927bc59a1ea53230297a6b5e68f02d3973" exitCode=0
Jul 10 14:10:43.500463 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:43.500069    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerDied","Data":"73f87a80deb7cb3b70066a5c87f5cf927bc59a1ea53230297a6b5e68f02d3973"}
Jul 10 14:10:43.500463 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:43.500129    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerStarted","Data":"9d6703cc4b6c3d2c72f8a209c23399c6d1e92dfe6c92afd7b0ec97216dfee3f0"}
Jul 10 14:10:44.506955 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:44.506920    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerStarted","Data":"7ad3cf4dfa2ad9b18a087ec63a67942278c0e38c3d226982055f9b92f9debc61"}
Jul 10 14:10:44.507315 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:44.507008    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerStarted","Data":"e7982b3c7cde666b3503f065fa3e406bccd2dae9dcf86a481c39420e6cfe786a"}
Jul 10 14:10:44.507315 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:44.507035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerStarted","Data":"228708ce754a62736aff20d960ea9a432afe4303f79bf96de4f5901024bc6640"}
Jul 10 14:10:44.507315 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:44.507063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerStarted","Data":"d3592896a3a0c8b354feed248efa009cc3fa08cc59a926c25ef545fc1a26125d"}
Jul 10 14:10:44.507315 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:44.507090    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerStarted","Data":"c7840dba7d0502927b69a8c9ccfead49631d730ad81d51ce2972248cf33682ca"}
Jul 10 14:10:44.507315 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:44.507113    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8998f81-303c-420a-88e3-a5e7031d074f","Type":"ContainerStarted","Data":"7ca9daf73533333ad86b7c2581d3418c8ddc904f2372133020d21f867a9f0b44"}
Jul 10 14:10:44.540443 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:44.540392    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.517416236 podStartE2EDuration="2.540373001s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.956765ms" isStatefulPod=false podCreationTimestamp="2026-07-10 14:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:10:44.538112476 +0000 UTC m=+116.163838407" watchObservedRunningTime="2026-07-10 14:10:44.540373001 +0000 UTC m=+116.166098933"
Jul 10 14:10:47.896317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:10:47.896274    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:11:21.308609 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:21.308574    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:21.438081 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:21.438045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:42.896139 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:42.896049    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:11:42.911323 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:42.911298    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:11:43.731456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:43.731424    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 14:11:45.870874 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:45.870843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:47.224409 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:47.224375    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:51.616329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:51.616293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:51.694495 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:51.694462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:54.269546 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:54.269505    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:54.688262 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:54.688227    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:58.088211 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:58.088177    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:11:58.188885 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:11:58.188852    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:00.174082 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:00.174046    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:00.893769 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:00.889916    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:02.289777 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:02.289741    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:02.393748 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:02.393676    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:04.802420 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:04.802393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:04.875808 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:04.875777    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:08.846610 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:08.846573    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:08.904784 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:08.904709    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:09.121248 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:09.121164    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:10.532780 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.532745    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b57t6"]
Jul 10 14:12:10.535825 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.535807    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.538842 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.538817    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 10 14:12:10.540072 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.540057    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ppv2m\""
Jul 10 14:12:10.540180 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.540082    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 10 14:12:10.546253 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.546227    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b57t6"]
Jul 10 14:12:10.596902 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.596867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w45w6\" (UniqueName: \"kubernetes.io/projected/93e52d89-9c1e-491c-8a1f-e36d0cf65462-kube-api-access-w45w6\") pod \"cert-manager-55b4d954d5-b57t6\" (UID: \"93e52d89-9c1e-491c-8a1f-e36d0cf65462\") " pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.597068 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.596918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93e52d89-9c1e-491c-8a1f-e36d0cf65462-bound-sa-token\") pod \"cert-manager-55b4d954d5-b57t6\" (UID: \"93e52d89-9c1e-491c-8a1f-e36d0cf65462\") " pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.698185 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.698147    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w45w6\" (UniqueName: \"kubernetes.io/projected/93e52d89-9c1e-491c-8a1f-e36d0cf65462-kube-api-access-w45w6\") pod \"cert-manager-55b4d954d5-b57t6\" (UID: \"93e52d89-9c1e-491c-8a1f-e36d0cf65462\") " pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.698376 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.698205    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93e52d89-9c1e-491c-8a1f-e36d0cf65462-bound-sa-token\") pod \"cert-manager-55b4d954d5-b57t6\" (UID: \"93e52d89-9c1e-491c-8a1f-e36d0cf65462\") " pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.707326 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.707298    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93e52d89-9c1e-491c-8a1f-e36d0cf65462-bound-sa-token\") pod \"cert-manager-55b4d954d5-b57t6\" (UID: \"93e52d89-9c1e-491c-8a1f-e36d0cf65462\") " pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.707393 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.707372    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w45w6\" (UniqueName: \"kubernetes.io/projected/93e52d89-9c1e-491c-8a1f-e36d0cf65462-kube-api-access-w45w6\") pod \"cert-manager-55b4d954d5-b57t6\" (UID: \"93e52d89-9c1e-491c-8a1f-e36d0cf65462\") " pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.814123 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.814048    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:10.861998 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.861965    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b57t6"
Jul 10 14:12:10.916027 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.915996    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:10.986259 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:10.986236    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b57t6"]
Jul 10 14:12:10.988514 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:12:10.988486    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93e52d89_9c1e_491c_8a1f_e36d0cf65462.slice/crio-eeac4e00882eaedbbaaca0b73bdbaadc5631137b1ecfd1b8fab882ec5d2f41a6 WatchSource:0}: Error finding container eeac4e00882eaedbbaaca0b73bdbaadc5631137b1ecfd1b8fab882ec5d2f41a6: Status 404 returned error can't find the container with id eeac4e00882eaedbbaaca0b73bdbaadc5631137b1ecfd1b8fab882ec5d2f41a6
Jul 10 14:12:11.813841 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:11.813807    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b57t6" event={"ID":"93e52d89-9c1e-491c-8a1f-e36d0cf65462","Type":"ContainerStarted","Data":"eeac4e00882eaedbbaaca0b73bdbaadc5631137b1ecfd1b8fab882ec5d2f41a6"}
Jul 10 14:12:13.040486 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.040452    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"]
Jul 10 14:12:13.043736 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.043700    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:13.046957 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.046930    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wwtm7\""
Jul 10 14:12:13.047077 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.047028    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 14:12:13.047223 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.047158    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 14:12:13.052984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.052935    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"]
Jul 10 14:12:13.119311 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.119281    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvsqv\" (UniqueName: \"kubernetes.io/projected/3a119259-101e-4f7a-b95f-7a0b022c9fe7-kube-api-access-fvsqv\") pod \"openshift-lws-operator-fd8ccff4c-hl92h\" (UID: \"3a119259-101e-4f7a-b95f-7a0b022c9fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:13.119494 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.119329    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a119259-101e-4f7a-b95f-7a0b022c9fe7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hl92h\" (UID: \"3a119259-101e-4f7a-b95f-7a0b022c9fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:13.219884 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.219800    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvsqv\" (UniqueName: \"kubernetes.io/projected/3a119259-101e-4f7a-b95f-7a0b022c9fe7-kube-api-access-fvsqv\") pod \"openshift-lws-operator-fd8ccff4c-hl92h\" (UID: \"3a119259-101e-4f7a-b95f-7a0b022c9fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:13.219884 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.219849    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a119259-101e-4f7a-b95f-7a0b022c9fe7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hl92h\" (UID: \"3a119259-101e-4f7a-b95f-7a0b022c9fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:13.220294 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.220267    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a119259-101e-4f7a-b95f-7a0b022c9fe7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hl92h\" (UID: \"3a119259-101e-4f7a-b95f-7a0b022c9fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:13.229024 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.229000    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvsqv\" (UniqueName: \"kubernetes.io/projected/3a119259-101e-4f7a-b95f-7a0b022c9fe7-kube-api-access-fvsqv\") pod \"openshift-lws-operator-fd8ccff4c-hl92h\" (UID: \"3a119259-101e-4f7a-b95f-7a0b022c9fe7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:13.356202 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:13.356165    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"
Jul 10 14:12:14.253273 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:14.253250    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h"]
Jul 10 14:12:14.256557 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:12:14.256533    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a119259_101e_4f7a_b95f_7a0b022c9fe7.slice/crio-12d38796833f3c176a5d72247a258646327da910575456cec2361edf48d4a2bd WatchSource:0}: Error finding container 12d38796833f3c176a5d72247a258646327da910575456cec2361edf48d4a2bd: Status 404 returned error can't find the container with id 12d38796833f3c176a5d72247a258646327da910575456cec2361edf48d4a2bd
Jul 10 14:12:14.828626 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:14.828586    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h" event={"ID":"3a119259-101e-4f7a-b95f-7a0b022c9fe7","Type":"ContainerStarted","Data":"12d38796833f3c176a5d72247a258646327da910575456cec2361edf48d4a2bd"}
Jul 10 14:12:14.829968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:14.829942    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b57t6" event={"ID":"93e52d89-9c1e-491c-8a1f-e36d0cf65462","Type":"ContainerStarted","Data":"4afb0ef552f76e4a659e5926b0360dc59df5e39c07533592a65e7f41a3525870"}
Jul 10 14:12:14.848067 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:14.848023    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-b57t6" podStartSLOduration=1.6520928700000002 podStartE2EDuration="4.848010817s" totalImagesPullingTime="3.195917947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:12:14.847732113 +0000 UTC m=+206.473458035" watchObservedRunningTime="2026-07-10 14:12:14.848010817 +0000 UTC m=+206.473736748"
Jul 10 14:12:16.839859 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:16.839829    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h" event={"ID":"3a119259-101e-4f7a-b95f-7a0b022c9fe7","Type":"ContainerStarted","Data":"992b03494620f6b51404cf0b7a8d63618849b27bfcc38cb22276f31ed26f6b28"}
Jul 10 14:12:16.858002 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:16.857954    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hl92h" podStartSLOduration=1.337693006 podStartE2EDuration="3.857937916s" totalImagesPullingTime="2.52024491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:12:16.857185046 +0000 UTC m=+208.482910979" watchObservedRunningTime="2026-07-10 14:12:16.857937916 +0000 UTC m=+208.483663848"
Jul 10 14:12:17.331966 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:17.331932    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:17.546570 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:17.546532    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:17.650830 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:17.650753    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:19.897910 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:19.897880    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:26.423356 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:26.423313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:31.217270 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:31.217227    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:31.314528 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:31.314494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:31.840080 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:31.840046    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.181387 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.181314    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.288667 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.288638    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.322305 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.322274    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"]
Jul 10 14:12:32.327919 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.327895    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.331620 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.331600    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 14:12:32.331777 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.331629    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 14:12:32.331777 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.331636    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cw2fh\""
Jul 10 14:12:32.331907 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.331883    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 14:12:32.332889 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.332874    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 14:12:32.341968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.341945    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"]
Jul 10 14:12:32.375659 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.375632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-manifests\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.375817 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.375670    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-tmp\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.375817 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.375687    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.375817 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.375713    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lswx9\" (UniqueName: \"kubernetes.io/projected/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-kube-api-access-lswx9\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.375817 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.375790    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.476459 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.476369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lswx9\" (UniqueName: \"kubernetes.io/projected/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-kube-api-access-lswx9\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.476459 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.476432    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.476647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.476489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-manifests\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.476647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.476612    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-tmp\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.476647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.476644    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.476931 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.476911    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-manifests\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.478884 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.478856    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-tmp\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.478993 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.478916    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.479061 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.479028    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.486694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.486674    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lswx9\" (UniqueName: \"kubernetes.io/projected/d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8-kube-api-access-lswx9\") pod \"opendatahub-operator-controller-manager-5d47b746d6-dwhl4\" (UID: \"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.487697 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.487664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.586552 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.586518    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.637656 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.637623    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:32.687607 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.687573    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.753966 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.753900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.773134 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.773108    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"]
Jul 10 14:12:32.776321 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:12:32.776297    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2104d9c_1677_4ac0_9ca8_ffec4d93cfd8.slice/crio-845ce017f01d522d60aa6f9f0bdc71d09ec78f788246c0b8ea784c10a4cf48ca WatchSource:0}: Error finding container 845ce017f01d522d60aa6f9f0bdc71d09ec78f788246c0b8ea784c10a4cf48ca: Status 404 returned error can't find the container with id 845ce017f01d522d60aa6f9f0bdc71d09ec78f788246c0b8ea784c10a4cf48ca
Jul 10 14:12:32.838059 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.838026    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:32.901374 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:32.901342    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4" event={"ID":"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8","Type":"ContainerStarted","Data":"845ce017f01d522d60aa6f9f0bdc71d09ec78f788246c0b8ea784c10a4cf48ca"}
Jul 10 14:12:36.905864 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:36.905832    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:36.921238 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:36.921211    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4" event={"ID":"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8","Type":"ContainerStarted","Data":"b9cc36d5116348a49eea2123ecac67045101a8727a9d37f08191df096a493192"}
Jul 10 14:12:37.926546 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:37.926510    2584 generic.go:356] "Generic (PLEG): container finished" podID="d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8" containerID="b9cc36d5116348a49eea2123ecac67045101a8727a9d37f08191df096a493192" exitCode=0
Jul 10 14:12:37.926921 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:37.926557    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4" event={"ID":"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8","Type":"ContainerDied","Data":"b9cc36d5116348a49eea2123ecac67045101a8727a9d37f08191df096a493192"}
Jul 10 14:12:38.932404 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:38.932369    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4" event={"ID":"d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8","Type":"ContainerStarted","Data":"9955d293e9ed704184a4b4c5e05c2357e59ad92be375d2916f8608046585402c"}
Jul 10 14:12:38.932811 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:38.932457    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:38.957681 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:38.957632    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4" podStartSLOduration=1.7366172899999999 podStartE2EDuration="6.957615153s" totalImagesPullingTime="4.176130671s" totalInitContainerRuntime="1.044867192s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:12:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:12:38.955631135 +0000 UTC m=+230.581357065" watchObservedRunningTime="2026-07-10 14:12:38.957615153 +0000 UTC m=+230.583341084"
Jul 10 14:12:44.135388 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:44.135360    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:44.251198 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:44.251166    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:49.938752 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:49.938705    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d47b746d6-dwhl4"
Jul 10 14:12:51.687592 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:51.687555    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:51.828257 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:51.828225    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:51.963368 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:51.963286    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:52.275251 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.275173    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"]
Jul 10 14:12:52.281133 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.281110    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.285824 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.285804    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 14:12:52.285942 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.285898    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 14:12:52.287538 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.287520    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 14:12:52.287607 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.287533    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-94r2r\""
Jul 10 14:12:52.287607 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.287544    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 14:12:52.314881 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.314851    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"]
Jul 10 14:12:52.445066 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.445033    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bbe80d9-d6a2-4f74-921e-76173b3ae289-tmp\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.445243 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.445106    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sxln\" (UniqueName: \"kubernetes.io/projected/8bbe80d9-d6a2-4f74-921e-76173b3ae289-kube-api-access-6sxln\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.445243 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.445146    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bbe80d9-d6a2-4f74-921e-76173b3ae289-tls-certs\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.546520 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.546427    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bbe80d9-d6a2-4f74-921e-76173b3ae289-tmp\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.546520 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.546485    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sxln\" (UniqueName: \"kubernetes.io/projected/8bbe80d9-d6a2-4f74-921e-76173b3ae289-kube-api-access-6sxln\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.546754 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.546543    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bbe80d9-d6a2-4f74-921e-76173b3ae289-tls-certs\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.548929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.548898    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bbe80d9-d6a2-4f74-921e-76173b3ae289-tmp\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.549076 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.549059    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bbe80d9-d6a2-4f74-921e-76173b3ae289-tls-certs\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.556413 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.556386    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sxln\" (UniqueName: \"kubernetes.io/projected/8bbe80d9-d6a2-4f74-921e-76173b3ae289-kube-api-access-6sxln\") pod \"kube-auth-proxy-8545bb46c8-9gsz5\" (UID: \"8bbe80d9-d6a2-4f74-921e-76173b3ae289\") " pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.589801 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.589776    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"
Jul 10 14:12:52.667153 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.667120    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"]
Jul 10 14:12:52.670630 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.669969    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.673351 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.672964    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 14:12:52.673533 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.673368    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-z7jnj\""
Jul 10 14:12:52.674339 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.674298    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 14:12:52.674450 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.674310    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 14:12:52.682227 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.682199    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"]
Jul 10 14:12:52.731580 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.731553    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5"]
Jul 10 14:12:52.733896 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:12:52.733874    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bbe80d9_d6a2_4f74_921e_76173b3ae289.slice/crio-a318565b52bd92797fb86f76cd64d1b282b7989a20915225ae00c582ccec4506 WatchSource:0}: Error finding container a318565b52bd92797fb86f76cd64d1b282b7989a20915225ae00c582ccec4506: Status 404 returned error can't find the container with id a318565b52bd92797fb86f76cd64d1b282b7989a20915225ae00c582ccec4506
Jul 10 14:12:52.850684 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.850654    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28ed8289-8054-4cc4-ac14-8e19410a6dbf-cert\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.850868 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.850705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/28ed8289-8054-4cc4-ac14-8e19410a6dbf-metrics-cert\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.850868 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.850799    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/28ed8289-8054-4cc4-ac14-8e19410a6dbf-manager-config\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.850942 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.850879    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbfzc\" (UniqueName: \"kubernetes.io/projected/28ed8289-8054-4cc4-ac14-8e19410a6dbf-kube-api-access-jbfzc\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.951299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.951265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28ed8289-8054-4cc4-ac14-8e19410a6dbf-cert\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.951299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.951303    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/28ed8289-8054-4cc4-ac14-8e19410a6dbf-metrics-cert\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.951491 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.951331    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/28ed8289-8054-4cc4-ac14-8e19410a6dbf-manager-config\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.951491 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.951377    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbfzc\" (UniqueName: \"kubernetes.io/projected/28ed8289-8054-4cc4-ac14-8e19410a6dbf-kube-api-access-jbfzc\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.952188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.952152    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/28ed8289-8054-4cc4-ac14-8e19410a6dbf-manager-config\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.952359 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.952221    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:52.953858 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.953838    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/28ed8289-8054-4cc4-ac14-8e19410a6dbf-cert\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.953943 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.953884    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/28ed8289-8054-4cc4-ac14-8e19410a6dbf-metrics-cert\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.966085 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.966064    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbfzc\" (UniqueName: \"kubernetes.io/projected/28ed8289-8054-4cc4-ac14-8e19410a6dbf-kube-api-access-jbfzc\") pod \"lws-controller-manager-5d5554d48-98lhq\" (UID: \"28ed8289-8054-4cc4-ac14-8e19410a6dbf\") " pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.985307 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.985285    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:52.986292 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:52.986270    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5" event={"ID":"8bbe80d9-d6a2-4f74-921e-76173b3ae289","Type":"ContainerStarted","Data":"a318565b52bd92797fb86f76cd64d1b282b7989a20915225ae00c582ccec4506"}
Jul 10 14:12:53.068340 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:53.068297    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:53.118872 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:53.118837    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"]
Jul 10 14:12:53.120832 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:12:53.120805    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ed8289_8054_4cc4_ac14_8e19410a6dbf.slice/crio-ac4005e96313afcdc4cc38e8fac2d3e1ef4547f87fd98a3008fa285419a3c3d5 WatchSource:0}: Error finding container ac4005e96313afcdc4cc38e8fac2d3e1ef4547f87fd98a3008fa285419a3c3d5: Status 404 returned error can't find the container with id ac4005e96313afcdc4cc38e8fac2d3e1ef4547f87fd98a3008fa285419a3c3d5
Jul 10 14:12:53.572460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:53.572429    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:53.791749 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:53.791689    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:53.868470 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:53.868394    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:53.991409 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:53.991374    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq" event={"ID":"28ed8289-8054-4cc4-ac14-8e19410a6dbf","Type":"ContainerStarted","Data":"ac4005e96313afcdc4cc38e8fac2d3e1ef4547f87fd98a3008fa285419a3c3d5"}
Jul 10 14:12:53.992347 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:53.992318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:55.144887 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:55.144860    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:12:56.003964 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:56.003932    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq" event={"ID":"28ed8289-8054-4cc4-ac14-8e19410a6dbf","Type":"ContainerStarted","Data":"b5f3dece598d8aa7e499b144026f054f32d4708ea7391dda2374749cfd1947df"}
Jul 10 14:12:56.004129 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:56.004001    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:12:56.022824 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:56.022782    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq" podStartSLOduration=2.078636753 podStartE2EDuration="4.0227682s" totalImagesPullingTime="1.944131447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:12:56.022536372 +0000 UTC m=+247.648262304" watchObservedRunningTime="2026-07-10 14:12:56.0227682 +0000 UTC m=+247.648494131"
Jul 10 14:12:58.013812 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:58.013698    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5" event={"ID":"8bbe80d9-d6a2-4f74-921e-76173b3ae289","Type":"ContainerStarted","Data":"d7c4e3ed31edc5ae37359345b66d4568c188f2cb92c8ea89207be61b1bdb86da"}
Jul 10 14:12:58.033556 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:12:58.033505    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8545bb46c8-9gsz5" podStartSLOduration=1.194096191 podStartE2EDuration="6.033489716s" totalImagesPullingTime="4.839393525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:12:58.031612702 +0000 UTC m=+249.657338634" watchObservedRunningTime="2026-07-10 14:12:58.033489716 +0000 UTC m=+249.659215689"
Jul 10 14:13:00.617680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:00.617645    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:00.748108 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:00.748076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:01.011269 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:01.011197    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:01.125257 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:01.125224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:01.230481 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:01.230448    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:04.020542 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:04.020508    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:04.338131 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:04.338086    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:04.459295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:04.459262    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:04.571397 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:04.571366    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:07.010813 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:07.010786    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5d5554d48-98lhq"
Jul 10 14:13:29.036796 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:29.036657    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:29.097033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:29.097001    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:39.121305 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:39.121251    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:39.184866 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:39.184835    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:40.967158 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:40.967105    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:41.034653 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:41.034624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:48.751427 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:48.751397    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:13:48.754684 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:48.754662    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 14:13:48.755696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:48.755675    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:13:53.245156 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:53.245080    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:53.818456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:53.818425    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:53.920196 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:53.920163    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:54.187896 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:54.187819    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:13:54.388334 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:13:54.388307    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:21.631526 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:21.631491    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:22.160393 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:22.160361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:26.977596 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:26.977562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:29.009546 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:29.009511    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:29.783053 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:29.783020    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:29.961059 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:29.961022    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:30.801052 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:30.801020    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:32.958930 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:32.958897    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:33.406362 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:33.406322    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:36.595151 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:36.595120    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:37.011755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.011666    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"]
Jul 10 14:14:37.013820 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.013797    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.016768 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.016746    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 14:14:37.016918 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.016877    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 14:14:37.017046 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.017031    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lxhdp\""
Jul 10 14:14:37.027040 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.027014    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"]
Jul 10 14:14:37.047580 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.047544    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrskn\" (UniqueName: \"kubernetes.io/projected/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-kube-api-access-rrskn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.047749 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.047585    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.148293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.148261    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrskn\" (UniqueName: \"kubernetes.io/projected/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-kube-api-access-rrskn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.148472 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.148300    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.148663 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.148647    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.158147 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.158120    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrskn\" (UniqueName: \"kubernetes.io/projected/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-kube-api-access-rrskn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.323595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.323564    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:37.456353 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.456327    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"]
Jul 10 14:14:37.458579 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:14:37.458549    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c2caba_d09d_47bf_b3cc_7198ef4c52d6.slice/crio-a1efdc658456453ec8a8987f7bbd3dac529e4e3af8353056294b91ba0bf95c8f WatchSource:0}: Error finding container a1efdc658456453ec8a8987f7bbd3dac529e4e3af8353056294b91ba0bf95c8f: Status 404 returned error can't find the container with id a1efdc658456453ec8a8987f7bbd3dac529e4e3af8353056294b91ba0bf95c8f
Jul 10 14:14:37.461114 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:37.461099    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:14:38.410018 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:38.409986    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x" event={"ID":"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6","Type":"ContainerStarted","Data":"a1efdc658456453ec8a8987f7bbd3dac529e4e3af8353056294b91ba0bf95c8f"}
Jul 10 14:14:38.571009 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:38.570965    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:38.665488 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:38.665402    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:38.961408 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:38.961331    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:39.063687 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:39.063651    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:42.363711 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:42.363567    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:42.463292 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:42.463262    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:45.308657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:45.308622    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:45.441831 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:45.441794    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x" event={"ID":"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6","Type":"ContainerStarted","Data":"b13413d084e2dd35852f34f25bf9b9ed31f7e7b0f757fbbbc5b3a66d737c01a2"}
Jul 10 14:14:45.442002 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:45.441927    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:45.463130 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:45.463084    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x" podStartSLOduration=2.355700846 podStartE2EDuration="9.463071547s" totalImagesPullingTime="7.107370701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:14:45.460525326 +0000 UTC m=+357.086251267" watchObservedRunningTime="2026-07-10 14:14:45.463071547 +0000 UTC m=+357.088797543"
Jul 10 14:14:46.565824 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:46.565793    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:46.666147 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:46.666114    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:47.247311 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.247272    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"]
Jul 10 14:14:47.249413 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.249392    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.252205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.252188    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 10 14:14:47.252320 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.252189    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 10 14:14:47.252619 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.252601    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-62cqs\""
Jul 10 14:14:47.267776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.267757    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"]
Jul 10 14:14:47.269113 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.269093    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:47.336199 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.336176    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.336323 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.336221    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9f9z\" (UniqueName: \"kubernetes.io/projected/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-kube-api-access-k9f9z\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.336323 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.336249    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.427390 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.427361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:47.437526 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.437505    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9f9z\" (UniqueName: \"kubernetes.io/projected/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-kube-api-access-k9f9z\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.437651 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.437542    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.437651 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.437585    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.438284 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.438260    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.439996 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.439977    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.464191 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.464164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9f9z\" (UniqueName: \"kubernetes.io/projected/b4f36db8-fb3a-46a5-a0d6-98d5abae8562-kube-api-access-k9f9z\") pod \"kuadrant-console-plugin-66b5774985-hns75\" (UID: \"b4f36db8-fb3a-46a5-a0d6-98d5abae8562\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.558869 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.558780    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"
Jul 10 14:14:47.710178 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:47.710157    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hns75"]
Jul 10 14:14:47.712749 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:14:47.712698    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4f36db8_fb3a_46a5_a0d6_98d5abae8562.slice/crio-f0ed24af956fb784abf0a2c6c6b29d06bb8b645925e785be43794bf428d97e2d WatchSource:0}: Error finding container f0ed24af956fb784abf0a2c6c6b29d06bb8b645925e785be43794bf428d97e2d: Status 404 returned error can't find the container with id f0ed24af956fb784abf0a2c6c6b29d06bb8b645925e785be43794bf428d97e2d
Jul 10 14:14:48.454027 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:48.453990    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75" event={"ID":"b4f36db8-fb3a-46a5-a0d6-98d5abae8562","Type":"ContainerStarted","Data":"f0ed24af956fb784abf0a2c6c6b29d06bb8b645925e785be43794bf428d97e2d"}
Jul 10 14:14:49.699518 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:49.699478    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:49.798118 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:49.798080    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:56.447677 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:56.447640    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:56.614477 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:56.614443    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:56.715988 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:56.715914    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:56.816554 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:56.816523    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:57.487567 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.487527    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"]
Jul 10 14:14:57.490866 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.490836    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.511974 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.511950    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"]
Jul 10 14:14:57.625861 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.625837    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn7xp\" (UniqueName: \"kubernetes.io/projected/81b0cee5-4e63-43fd-82ee-4e962159a854-kube-api-access-dn7xp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.626022 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.625894    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b0cee5-4e63-43fd-82ee-4e962159a854-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.639161 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.639129    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:57.726513 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.726486    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b0cee5-4e63-43fd-82ee-4e962159a854-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.726662 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.726544    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn7xp\" (UniqueName: \"kubernetes.io/projected/81b0cee5-4e63-43fd-82ee-4e962159a854-kube-api-access-dn7xp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.726950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.726930    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b0cee5-4e63-43fd-82ee-4e962159a854-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.739497 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.739436    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:57.742004 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.741985    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn7xp\" (UniqueName: \"kubernetes.io/projected/81b0cee5-4e63-43fd-82ee-4e962159a854-kube-api-access-dn7xp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.800578 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.800547    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:57.943163 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.943082    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:57.943163 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:57.943147    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"]
Jul 10 14:14:57.946633 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:14:57.946611    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b0cee5_4e63_43fd_82ee_4e962159a854.slice/crio-8d0e5ac6948e32593aba04f7a72cf697aa7c829c5f778567c79ad002912aba31 WatchSource:0}: Error finding container 8d0e5ac6948e32593aba04f7a72cf697aa7c829c5f778567c79ad002912aba31: Status 404 returned error can't find the container with id 8d0e5ac6948e32593aba04f7a72cf697aa7c829c5f778567c79ad002912aba31
Jul 10 14:14:58.057094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.057066    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:14:58.327696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.327666    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"]
Jul 10 14:14:58.331732 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.331693    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"]
Jul 10 14:14:58.344299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.344273    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"]
Jul 10 14:14:58.344657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.344625    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" containerName="manager" containerID="cri-o://b13413d084e2dd35852f34f25bf9b9ed31f7e7b0f757fbbbc5b3a66d737c01a2" gracePeriod=2
Jul 10 14:14:58.346696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.346671    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"]
Jul 10 14:14:58.347241 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.347215    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:58.354465 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.354394    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"]
Jul 10 14:14:58.354914 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.354874    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" containerName="manager"
Jul 10 14:14:58.354914 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.354913    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" containerName="manager"
Jul 10 14:14:58.355045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.354955    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" containerName="manager"
Jul 10 14:14:58.355045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.354965    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" containerName="manager"
Jul 10 14:14:58.355045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.355024    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" containerName="manager"
Jul 10 14:14:58.355045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.355032    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" containerName="manager"
Jul 10 14:14:58.359432 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.359414    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.361896 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.361866    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:58.379123 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.379094    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"]
Jul 10 14:14:58.397008 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.396871    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"]
Jul 10 14:14:58.399695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.399670    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.407472 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.407442    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:58.425152 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.425128    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"]
Jul 10 14:14:58.432904 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.432869    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqxg6\" (UniqueName: \"kubernetes.io/projected/aa081794-15a4-45a9-8e10-b54dc57fc255-kube-api-access-gqxg6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8x4p\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.433004 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.432946    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa081794-15a4-45a9-8e10-b54dc57fc255-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8x4p\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.502943 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.502886    2584 generic.go:356] "Generic (PLEG): container finished" podID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" containerID="b13413d084e2dd35852f34f25bf9b9ed31f7e7b0f757fbbbc5b3a66d737c01a2" exitCode=0
Jul 10 14:14:58.504942 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.504921    2584 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts" container="manager"
Jul 10 14:14:58.507537 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.507514    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:58.510417 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.510393    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:58.534391 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.534193    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxg6\" (UniqueName: \"kubernetes.io/projected/aa081794-15a4-45a9-8e10-b54dc57fc255-kube-api-access-gqxg6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8x4p\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.534391 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.534241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76b2ed4-a400-4827-a7dd-154c76111749-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znng7\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.534391 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.534328    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa081794-15a4-45a9-8e10-b54dc57fc255-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8x4p\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.534623 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.534404    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wdhw\" (UniqueName: \"kubernetes.io/projected/b76b2ed4-a400-4827-a7dd-154c76111749-kube-api-access-8wdhw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znng7\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.534778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.534752    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa081794-15a4-45a9-8e10-b54dc57fc255-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8x4p\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.545625 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.545607    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqxg6\" (UniqueName: \"kubernetes.io/projected/aa081794-15a4-45a9-8e10-b54dc57fc255-kube-api-access-gqxg6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8x4p\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.585862 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.585816    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:58.588458 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.588435    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:58.590646 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.590626    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:58.635113 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.635088    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wdhw\" (UniqueName: \"kubernetes.io/projected/b76b2ed4-a400-4827-a7dd-154c76111749-kube-api-access-8wdhw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znng7\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.635270 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.635171    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76b2ed4-a400-4827-a7dd-154c76111749-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znng7\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.635568 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.635543    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76b2ed4-a400-4827-a7dd-154c76111749-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znng7\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.650504 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.650483    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wdhw\" (UniqueName: \"kubernetes.io/projected/b76b2ed4-a400-4827-a7dd-154c76111749-kube-api-access-8wdhw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-znng7\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.706456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.706427    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:14:58.715777 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.715755    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:14:58.736005 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.735983    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-extensions-socket-volume\") pod \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") "
Jul 10 14:14:58.736124 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.736059    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrskn\" (UniqueName: \"kubernetes.io/projected/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-kube-api-access-rrskn\") pod \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\" (UID: \"f7c2caba-d09d-47bf-b3cc-7198ef4c52d6\") "
Jul 10 14:14:58.736387 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.736358    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" (UID: "f7c2caba-d09d-47bf-b3cc-7198ef4c52d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:14:58.738319 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.738292    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-kube-api-access-rrskn" (OuterVolumeSpecName: "kube-api-access-rrskn") pod "f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" (UID: "f7c2caba-d09d-47bf-b3cc-7198ef4c52d6"). InnerVolumeSpecName "kube-api-access-rrskn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:14:58.837407 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.837350    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-extensions-socket-volume\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:14:58.837407 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.837377    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrskn\" (UniqueName: \"kubernetes.io/projected/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6-kube-api-access-rrskn\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:14:58.878825 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.878789    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"]
Jul 10 14:14:58.900764 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.900742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"]
Jul 10 14:14:58.920474 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.920443    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v467x\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:58.921362 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:58.921342    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c2caba-d09d-47bf-b3cc-7198ef4c52d6" path="/var/lib/kubelet/pods/f7c2caba-d09d-47bf-b3cc-7198ef4c52d6/volumes"
Jul 10 14:14:58.922994 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:14:58.922973    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:14:59.511443 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:59.511409    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v467x"
Jul 10 14:14:59.511903 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:59.511431    2584 scope.go:117] "RemoveContainer" containerID="b13413d084e2dd35852f34f25bf9b9ed31f7e7b0f757fbbbc5b3a66d737c01a2"
Jul 10 14:14:59.513962 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:14:59.513854    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" containerName="manager" containerID="cri-o://19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0" gracePeriod=2
Jul 10 14:15:00.531488 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:00.531451    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:00.646327 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:00.646291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:05.750477 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:05.750439    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:05.888355 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:05.888286    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:09.337626 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:09.337572    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:09.393202 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:09.393158    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:12.442254 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:15:12.442215    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76b2ed4_a400_4827_a7dd_154c76111749.slice/crio-da4d66cfded491797e7a2a3f0506131297365de0c313f02c705d0e8b39fa34c0 WatchSource:0}: Error finding container da4d66cfded491797e7a2a3f0506131297365de0c313f02c705d0e8b39fa34c0: Status 404 returned error can't find the container with id da4d66cfded491797e7a2a3f0506131297365de0c313f02c705d0e8b39fa34c0
Jul 10 14:15:12.443375 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:15:12.443336    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa081794_15a4_45a9_8e10_b54dc57fc255.slice/crio-33c16c0bb256cbfbce0846784fa1bcdb059a2a02ec499794d3153bb204047557 WatchSource:0}: Error finding container 33c16c0bb256cbfbce0846784fa1bcdb059a2a02ec499794d3153bb204047557: Status 404 returned error can't find the container with id 33c16c0bb256cbfbce0846784fa1bcdb059a2a02ec499794d3153bb204047557
Jul 10 14:15:12.563899 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.563876    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:15:12.566217 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:15:12.566190    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:15:12.579378 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.579354    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7" event={"ID":"b76b2ed4-a400-4827-a7dd-154c76111749","Type":"ContainerStarted","Data":"2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7"}
Jul 10 14:15:12.579490 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.579413    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:15:12.579490 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.579439    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7" event={"ID":"b76b2ed4-a400-4827-a7dd-154c76111749","Type":"ContainerStarted","Data":"da4d66cfded491797e7a2a3f0506131297365de0c313f02c705d0e8b39fa34c0"}
Jul 10 14:15:12.580647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.580627    2584 generic.go:356] "Generic (PLEG): container finished" podID="81b0cee5-4e63-43fd-82ee-4e962159a854" containerID="19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0" exitCode=0
Jul 10 14:15:12.580755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.580674    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:15:12.580755 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.580710    2584 scope.go:117] "RemoveContainer" containerID="19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0"
Jul 10 14:15:12.582035 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:15:12.582004    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:15:12.582476 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.582446    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p" event={"ID":"aa081794-15a4-45a9-8e10-b54dc57fc255","Type":"ContainerStarted","Data":"2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b"}
Jul 10 14:15:12.582579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.582498    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p" event={"ID":"aa081794-15a4-45a9-8e10-b54dc57fc255","Type":"ContainerStarted","Data":"33c16c0bb256cbfbce0846784fa1bcdb059a2a02ec499794d3153bb204047557"}
Jul 10 14:15:12.582579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.582539    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:15:12.584782 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.584761    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75" event={"ID":"b4f36db8-fb3a-46a5-a0d6-98d5abae8562","Type":"ContainerStarted","Data":"ba3ee9706282bfe7574271667b5ac0a5bf016ec23e1a7815ed6a86d67931e878"}
Jul 10 14:15:12.588729 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.588700    2584 scope.go:117] "RemoveContainer" containerID="19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0"
Jul 10 14:15:12.588970 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:15:12.588954    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0\": container with ID starting with 19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0 not found: ID does not exist" containerID="19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0"
Jul 10 14:15:12.589031 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.588975    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0"} err="failed to get container status \"19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0\": rpc error: code = NotFound desc = could not find container \"19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0\": container with ID starting with 19c9a90d3fc3442cadff701075079c8e2da6af231cab2fdc8c108a7eedd907d0 not found: ID does not exist"
Jul 10 14:15:12.602619 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.602583    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7" podStartSLOduration=14.602572584 podStartE2EDuration="14.602572584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:15:12.600819248 +0000 UTC m=+384.226545196" watchObservedRunningTime="2026-07-10 14:15:12.602572584 +0000 UTC m=+384.228298513"
Jul 10 14:15:12.617787 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.617751    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hns75" podStartSLOduration=0.827892513 podStartE2EDuration="25.617736236s" totalImagesPullingTime="24.789843723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:14:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:15:12.616515136 +0000 UTC m=+384.242241067" watchObservedRunningTime="2026-07-10 14:15:12.617736236 +0000 UTC m=+384.243462171"
Jul 10 14:15:12.639241 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:15:12.639209    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:15:12.759778 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.759696    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn7xp\" (UniqueName: \"kubernetes.io/projected/81b0cee5-4e63-43fd-82ee-4e962159a854-kube-api-access-dn7xp\") pod \"81b0cee5-4e63-43fd-82ee-4e962159a854\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") "
Jul 10 14:15:12.759903 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.759797    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b0cee5-4e63-43fd-82ee-4e962159a854-extensions-socket-volume\") pod \"81b0cee5-4e63-43fd-82ee-4e962159a854\" (UID: \"81b0cee5-4e63-43fd-82ee-4e962159a854\") "
Jul 10 14:15:12.760197 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.760165    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b0cee5-4e63-43fd-82ee-4e962159a854-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "81b0cee5-4e63-43fd-82ee-4e962159a854" (UID: "81b0cee5-4e63-43fd-82ee-4e962159a854"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:15:12.761712 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.761691    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b0cee5-4e63-43fd-82ee-4e962159a854-kube-api-access-dn7xp" (OuterVolumeSpecName: "kube-api-access-dn7xp") pod "81b0cee5-4e63-43fd-82ee-4e962159a854" (UID: "81b0cee5-4e63-43fd-82ee-4e962159a854"). InnerVolumeSpecName "kube-api-access-dn7xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:15:12.860960 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.860913    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn7xp\" (UniqueName: \"kubernetes.io/projected/81b0cee5-4e63-43fd-82ee-4e962159a854-kube-api-access-dn7xp\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:15:12.860960 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.860954    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b0cee5-4e63-43fd-82ee-4e962159a854-extensions-socket-volume\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:15:12.894358 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:15:12.894329    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-q6kts\" is forbidden: User \"system:node:ip-10-0-143-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-123.ec2.internal' and this object" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-q6kts"
Jul 10 14:15:12.926380 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:12.926331    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b0cee5-4e63-43fd-82ee-4e962159a854" path="/var/lib/kubelet/pods/81b0cee5-4e63-43fd-82ee-4e962159a854/volumes"
Jul 10 14:15:23.592283 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.592248    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:15:23.592766 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.592585    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:15:23.615268 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.615223    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p" podStartSLOduration=25.615209959 podStartE2EDuration="25.615209959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:15:12.636989078 +0000 UTC m=+384.262715008" watchObservedRunningTime="2026-07-10 14:15:23.615209959 +0000 UTC m=+395.240935890"
Jul 10 14:15:23.665416 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.665385    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"]
Jul 10 14:15:23.665626 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.665600    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7" podUID="b76b2ed4-a400-4827-a7dd-154c76111749" containerName="manager" containerID="cri-o://2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7" gracePeriod=10
Jul 10 14:15:23.857344 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.857268    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"]
Jul 10 14:15:23.865009 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.864988    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:23.879331 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.879308    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"]
Jul 10 14:15:23.937373 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.937352    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:15:23.941533 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.941506    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g87r2\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:23.941643 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:23.941552    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vjn8\" (UniqueName: \"kubernetes.io/projected/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-kube-api-access-8vjn8\") pod \"kuadrant-operator-controller-manager-86878c769f-g87r2\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:24.042547 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.042521    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wdhw\" (UniqueName: \"kubernetes.io/projected/b76b2ed4-a400-4827-a7dd-154c76111749-kube-api-access-8wdhw\") pod \"b76b2ed4-a400-4827-a7dd-154c76111749\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") "
Jul 10 14:15:24.042677 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.042622    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76b2ed4-a400-4827-a7dd-154c76111749-extensions-socket-volume\") pod \"b76b2ed4-a400-4827-a7dd-154c76111749\" (UID: \"b76b2ed4-a400-4827-a7dd-154c76111749\") "
Jul 10 14:15:24.042735 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.042701    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g87r2\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:24.042780 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.042745    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vjn8\" (UniqueName: \"kubernetes.io/projected/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-kube-api-access-8vjn8\") pod \"kuadrant-operator-controller-manager-86878c769f-g87r2\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:24.043139 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.043106    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76b2ed4-a400-4827-a7dd-154c76111749-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b76b2ed4-a400-4827-a7dd-154c76111749" (UID: "b76b2ed4-a400-4827-a7dd-154c76111749"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:15:24.043251 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.043179    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g87r2\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:24.044781 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.044763    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76b2ed4-a400-4827-a7dd-154c76111749-kube-api-access-8wdhw" (OuterVolumeSpecName: "kube-api-access-8wdhw") pod "b76b2ed4-a400-4827-a7dd-154c76111749" (UID: "b76b2ed4-a400-4827-a7dd-154c76111749"). InnerVolumeSpecName "kube-api-access-8wdhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:15:24.053891 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.053870    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vjn8\" (UniqueName: \"kubernetes.io/projected/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-kube-api-access-8vjn8\") pod \"kuadrant-operator-controller-manager-86878c769f-g87r2\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:24.143456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.143397    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76b2ed4-a400-4827-a7dd-154c76111749-extensions-socket-volume\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:15:24.143456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.143419    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wdhw\" (UniqueName: \"kubernetes.io/projected/b76b2ed4-a400-4827-a7dd-154c76111749-kube-api-access-8wdhw\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:15:24.174348 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.174326    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:24.309687 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.306020    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"]
Jul 10 14:15:24.313607 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:15:24.313568    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f8214c_150a_4fc2_8e0d_6080ffd1cacb.slice/crio-6dfaf6a61602b3ca4da37e678920d2061b93784323b4d5eee9b935fc4da109f3 WatchSource:0}: Error finding container 6dfaf6a61602b3ca4da37e678920d2061b93784323b4d5eee9b935fc4da109f3: Status 404 returned error can't find the container with id 6dfaf6a61602b3ca4da37e678920d2061b93784323b4d5eee9b935fc4da109f3
Jul 10 14:15:24.638735 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.638683    2584 generic.go:356] "Generic (PLEG): container finished" podID="b76b2ed4-a400-4827-a7dd-154c76111749" containerID="2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7" exitCode=0
Jul 10 14:15:24.639155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.638748    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"
Jul 10 14:15:24.639155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.638748    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7" event={"ID":"b76b2ed4-a400-4827-a7dd-154c76111749","Type":"ContainerDied","Data":"2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7"}
Jul 10 14:15:24.639155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.638798    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7" event={"ID":"b76b2ed4-a400-4827-a7dd-154c76111749","Type":"ContainerDied","Data":"da4d66cfded491797e7a2a3f0506131297365de0c313f02c705d0e8b39fa34c0"}
Jul 10 14:15:24.639155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.638820    2584 scope.go:117] "RemoveContainer" containerID="2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7"
Jul 10 14:15:24.640492 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.640470    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2" event={"ID":"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb","Type":"ContainerStarted","Data":"d74610a4c753d21744aa0cecb9e40277879070b1b437fe9a3146d5e12ed4206c"}
Jul 10 14:15:24.640679 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.640506    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2" event={"ID":"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb","Type":"ContainerStarted","Data":"6dfaf6a61602b3ca4da37e678920d2061b93784323b4d5eee9b935fc4da109f3"}
Jul 10 14:15:24.640679 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.640582    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:24.648695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.648677    2584 scope.go:117] "RemoveContainer" containerID="2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7"
Jul 10 14:15:24.649041 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:15:24.649015    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7\": container with ID starting with 2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7 not found: ID does not exist" containerID="2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7"
Jul 10 14:15:24.649137 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.649106    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7"} err="failed to get container status \"2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7\": rpc error: code = NotFound desc = could not find container \"2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7\": container with ID starting with 2398bbe263b096f8dd07654003e832ec235afff04ea8e29e9667af88a0d654b7 not found: ID does not exist"
Jul 10 14:15:24.663456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.663417    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2" podStartSLOduration=1.663406584 podStartE2EDuration="1.663406584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:15:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:15:24.662589617 +0000 UTC m=+396.288315549" watchObservedRunningTime="2026-07-10 14:15:24.663406584 +0000 UTC m=+396.289132512"
Jul 10 14:15:24.676735 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.676701    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"]
Jul 10 14:15:24.683272 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.683253    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-znng7"]
Jul 10 14:15:24.921957 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:24.921886    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b76b2ed4-a400-4827-a7dd-154c76111749" path="/var/lib/kubelet/pods/b76b2ed4-a400-4827-a7dd-154c76111749/volumes"
Jul 10 14:15:35.648693 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:35.648656    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:15:35.704348 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:35.704316    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"]
Jul 10 14:15:35.704614 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:35.704560    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p" podUID="aa081794-15a4-45a9-8e10-b54dc57fc255" containerName="manager" containerID="cri-o://2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b" gracePeriod=10
Jul 10 14:15:35.950213 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:35.950190    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:15:36.038174 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.038143    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqxg6\" (UniqueName: \"kubernetes.io/projected/aa081794-15a4-45a9-8e10-b54dc57fc255-kube-api-access-gqxg6\") pod \"aa081794-15a4-45a9-8e10-b54dc57fc255\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") "
Jul 10 14:15:36.038339 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.038190    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa081794-15a4-45a9-8e10-b54dc57fc255-extensions-socket-volume\") pod \"aa081794-15a4-45a9-8e10-b54dc57fc255\" (UID: \"aa081794-15a4-45a9-8e10-b54dc57fc255\") "
Jul 10 14:15:36.038563 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.038541    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa081794-15a4-45a9-8e10-b54dc57fc255-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa081794-15a4-45a9-8e10-b54dc57fc255" (UID: "aa081794-15a4-45a9-8e10-b54dc57fc255"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:15:36.040219 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.040197    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa081794-15a4-45a9-8e10-b54dc57fc255-kube-api-access-gqxg6" (OuterVolumeSpecName: "kube-api-access-gqxg6") pod "aa081794-15a4-45a9-8e10-b54dc57fc255" (UID: "aa081794-15a4-45a9-8e10-b54dc57fc255"). InnerVolumeSpecName "kube-api-access-gqxg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:15:36.138939 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.138902    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa081794-15a4-45a9-8e10-b54dc57fc255-extensions-socket-volume\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:15:36.138939 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.138933    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqxg6\" (UniqueName: \"kubernetes.io/projected/aa081794-15a4-45a9-8e10-b54dc57fc255-kube-api-access-gqxg6\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:15:36.697816 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.697784    2584 generic.go:356] "Generic (PLEG): container finished" podID="aa081794-15a4-45a9-8e10-b54dc57fc255" containerID="2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b" exitCode=0
Jul 10 14:15:36.698236 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.697841    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"
Jul 10 14:15:36.698236 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.697879    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p" event={"ID":"aa081794-15a4-45a9-8e10-b54dc57fc255","Type":"ContainerDied","Data":"2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b"}
Jul 10 14:15:36.698236 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.697927    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p" event={"ID":"aa081794-15a4-45a9-8e10-b54dc57fc255","Type":"ContainerDied","Data":"33c16c0bb256cbfbce0846784fa1bcdb059a2a02ec499794d3153bb204047557"}
Jul 10 14:15:36.698236 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.697946    2584 scope.go:117] "RemoveContainer" containerID="2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b"
Jul 10 14:15:36.706570 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.706428    2584 scope.go:117] "RemoveContainer" containerID="2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b"
Jul 10 14:15:36.706702 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:15:36.706682    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b\": container with ID starting with 2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b not found: ID does not exist" containerID="2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b"
Jul 10 14:15:36.706776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.706711    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b"} err="failed to get container status \"2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b\": rpc error: code = NotFound desc = could not find container \"2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b\": container with ID starting with 2665431149cc7738ec001e83efd6933457f13408476ff03342f5f9bbbf6c652b not found: ID does not exist"
Jul 10 14:15:36.724132 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.724028    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"]
Jul 10 14:15:36.728335 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.728315    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8x4p"]
Jul 10 14:15:36.920794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:36.920765    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa081794-15a4-45a9-8e10-b54dc57fc255" path="/var/lib/kubelet/pods/aa081794-15a4-45a9-8e10-b54dc57fc255/volumes"
Jul 10 14:15:40.085253 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:40.085216    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:40.307665 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:40.307633    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:52.777395 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:52.777363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:52.907090 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:52.907056    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:52.965481 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:52.965447    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:53.609862 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.609824    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rrg5p"]
Jul 10 14:15:53.610338 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.610313    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76b2ed4-a400-4827-a7dd-154c76111749" containerName="manager"
Jul 10 14:15:53.610466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.610394    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76b2ed4-a400-4827-a7dd-154c76111749" containerName="manager"
Jul 10 14:15:53.610466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.610412    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa081794-15a4-45a9-8e10-b54dc57fc255" containerName="manager"
Jul 10 14:15:53.610466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.610421    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa081794-15a4-45a9-8e10-b54dc57fc255" containerName="manager"
Jul 10 14:15:53.610604 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.610531    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b76b2ed4-a400-4827-a7dd-154c76111749" containerName="manager"
Jul 10 14:15:53.610604 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.610548    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa081794-15a4-45a9-8e10-b54dc57fc255" containerName="manager"
Jul 10 14:15:53.644667 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.644631    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:53.673350 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.673314    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rrg5p"]
Jul 10 14:15:53.673514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.673416    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rrg5p"
Jul 10 14:15:53.676311 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.676293    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cmfx9\""
Jul 10 14:15:53.779121 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.779088    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7rdzh"]
Jul 10 14:15:53.781446 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.781424    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7rdzh"
Jul 10 14:15:53.787437 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.787407    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvxt5\" (UniqueName: \"kubernetes.io/projected/eede577b-8453-48ce-84d4-78d48b357850-kube-api-access-vvxt5\") pod \"authorino-58cd955675-rrg5p\" (UID: \"eede577b-8453-48ce-84d4-78d48b357850\") " pod="kuadrant-system/authorino-58cd955675-rrg5p"
Jul 10 14:15:53.789435 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.789416    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7rdzh"]
Jul 10 14:15:53.823868 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.823833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:15:53.888189 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.888107    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6zsj\" (UniqueName: \"kubernetes.io/projected/9b05f767-323d-47db-b0bd-16baf6b16ef2-kube-api-access-t6zsj\") pod \"authorino-75d556bd77-7rdzh\" (UID: \"9b05f767-323d-47db-b0bd-16baf6b16ef2\") " pod="kuadrant-system/authorino-75d556bd77-7rdzh"
Jul 10 14:15:53.888189 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.888173    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvxt5\" (UniqueName: \"kubernetes.io/projected/eede577b-8453-48ce-84d4-78d48b357850-kube-api-access-vvxt5\") pod \"authorino-58cd955675-rrg5p\" (UID: \"eede577b-8453-48ce-84d4-78d48b357850\") " pod="kuadrant-system/authorino-58cd955675-rrg5p"
Jul 10 14:15:53.896883 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.896853    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvxt5\" (UniqueName: \"kubernetes.io/projected/eede577b-8453-48ce-84d4-78d48b357850-kube-api-access-vvxt5\") pod \"authorino-58cd955675-rrg5p\" (UID: \"eede577b-8453-48ce-84d4-78d48b357850\") " pod="kuadrant-system/authorino-58cd955675-rrg5p"
Jul 10 14:15:53.981934 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.981892    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rrg5p"
Jul 10 14:15:53.989427 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.989401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6zsj\" (UniqueName: \"kubernetes.io/projected/9b05f767-323d-47db-b0bd-16baf6b16ef2-kube-api-access-t6zsj\") pod \"authorino-75d556bd77-7rdzh\" (UID: \"9b05f767-323d-47db-b0bd-16baf6b16ef2\") " pod="kuadrant-system/authorino-75d556bd77-7rdzh"
Jul 10 14:15:53.998274 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:53.998248    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6zsj\" (UniqueName: \"kubernetes.io/projected/9b05f767-323d-47db-b0bd-16baf6b16ef2-kube-api-access-t6zsj\") pod \"authorino-75d556bd77-7rdzh\" (UID: \"9b05f767-323d-47db-b0bd-16baf6b16ef2\") " pod="kuadrant-system/authorino-75d556bd77-7rdzh"
Jul 10 14:15:54.090019 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:54.089988    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7rdzh"
Jul 10 14:15:54.113153 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:54.113126    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rrg5p"]
Jul 10 14:15:54.115675 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:15:54.115647    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeede577b_8453_48ce_84d4_78d48b357850.slice/crio-e0da9fef348cf74506b61ce2e74d059e297b5bf6d2d4e5ff5d5f1825a09c82fc WatchSource:0}: Error finding container e0da9fef348cf74506b61ce2e74d059e297b5bf6d2d4e5ff5d5f1825a09c82fc: Status 404 returned error can't find the container with id e0da9fef348cf74506b61ce2e74d059e297b5bf6d2d4e5ff5d5f1825a09c82fc
Jul 10 14:15:54.218449 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:54.218420    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7rdzh"]
Jul 10 14:15:54.220728 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:15:54.220682    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b05f767_323d_47db_b0bd_16baf6b16ef2.slice/crio-eaeca54aaaace04ba7d29402e3a77922e238dcd3340df1b91826e04d54fdb9e9 WatchSource:0}: Error finding container eaeca54aaaace04ba7d29402e3a77922e238dcd3340df1b91826e04d54fdb9e9: Status 404 returned error can't find the container with id eaeca54aaaace04ba7d29402e3a77922e238dcd3340df1b91826e04d54fdb9e9
Jul 10 14:15:54.780249 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:54.780214    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7rdzh" event={"ID":"9b05f767-323d-47db-b0bd-16baf6b16ef2","Type":"ContainerStarted","Data":"eaeca54aaaace04ba7d29402e3a77922e238dcd3340df1b91826e04d54fdb9e9"}
Jul 10 14:15:54.781318 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:54.781298    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rrg5p" event={"ID":"eede577b-8453-48ce-84d4-78d48b357850","Type":"ContainerStarted","Data":"e0da9fef348cf74506b61ce2e74d059e297b5bf6d2d4e5ff5d5f1825a09c82fc"}
Jul 10 14:15:57.806964 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:57.806925    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rrg5p" event={"ID":"eede577b-8453-48ce-84d4-78d48b357850","Type":"ContainerStarted","Data":"14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384"}
Jul 10 14:15:57.808329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:57.808304    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7rdzh" event={"ID":"9b05f767-323d-47db-b0bd-16baf6b16ef2","Type":"ContainerStarted","Data":"91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b"}
Jul 10 14:15:57.823635 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:57.823589    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rrg5p" podStartSLOduration=1.3281058319999999 podStartE2EDuration="4.823576851s" totalImagesPullingTime="3.495471019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:15:57.822871993 +0000 UTC m=+429.448597923" watchObservedRunningTime="2026-07-10 14:15:57.823576851 +0000 UTC m=+429.449302780"
Jul 10 14:15:57.838474 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:57.838418    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7rdzh" podStartSLOduration=1.4579111 podStartE2EDuration="4.838403771s" totalImagesPullingTime="3.380492671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:15:57.837416617 +0000 UTC m=+429.463142549" watchObservedRunningTime="2026-07-10 14:15:57.838403771 +0000 UTC m=+429.464129702"
Jul 10 14:15:57.860745 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:57.860654    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rrg5p"]
Jul 10 14:15:59.816656 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:15:59.816619    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rrg5p" podUID="eede577b-8453-48ce-84d4-78d48b357850" containerName="authorino" containerID="cri-o://14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384" gracePeriod=30
Jul 10 14:16:00.049142 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.049117    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rrg5p"
Jul 10 14:16:00.147209 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.147118    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvxt5\" (UniqueName: \"kubernetes.io/projected/eede577b-8453-48ce-84d4-78d48b357850-kube-api-access-vvxt5\") pod \"eede577b-8453-48ce-84d4-78d48b357850\" (UID: \"eede577b-8453-48ce-84d4-78d48b357850\") "
Jul 10 14:16:00.149152 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.149129    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eede577b-8453-48ce-84d4-78d48b357850-kube-api-access-vvxt5" (OuterVolumeSpecName: "kube-api-access-vvxt5") pod "eede577b-8453-48ce-84d4-78d48b357850" (UID: "eede577b-8453-48ce-84d4-78d48b357850"). InnerVolumeSpecName "kube-api-access-vvxt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:16:00.248805 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.248766    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvxt5\" (UniqueName: \"kubernetes.io/projected/eede577b-8453-48ce-84d4-78d48b357850-kube-api-access-vvxt5\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:16:00.821202 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.821164    2584 generic.go:356] "Generic (PLEG): container finished" podID="eede577b-8453-48ce-84d4-78d48b357850" containerID="14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384" exitCode=0
Jul 10 14:16:00.821632 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.821207    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rrg5p"
Jul 10 14:16:00.821632 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.821253    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rrg5p" event={"ID":"eede577b-8453-48ce-84d4-78d48b357850","Type":"ContainerDied","Data":"14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384"}
Jul 10 14:16:00.821632 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.821297    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rrg5p" event={"ID":"eede577b-8453-48ce-84d4-78d48b357850","Type":"ContainerDied","Data":"e0da9fef348cf74506b61ce2e74d059e297b5bf6d2d4e5ff5d5f1825a09c82fc"}
Jul 10 14:16:00.821632 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.821317    2584 scope.go:117] "RemoveContainer" containerID="14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384"
Jul 10 14:16:00.829333 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.829317    2584 scope.go:117] "RemoveContainer" containerID="14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384"
Jul 10 14:16:00.829584 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:16:00.829565    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384\": container with ID starting with 14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384 not found: ID does not exist" containerID="14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384"
Jul 10 14:16:00.829650 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.829590    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384"} err="failed to get container status \"14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384\": rpc error: code = NotFound desc = could not find container \"14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384\": container with ID starting with 14b9aa5046689d1137ae2149a3aac896f91b2dc9ffb7f3a09c1730fd94311384 not found: ID does not exist"
Jul 10 14:16:00.842183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.842107    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rrg5p"]
Jul 10 14:16:00.843895 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.843876    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rrg5p"]
Jul 10 14:16:00.919877 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:00.919844    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eede577b-8453-48ce-84d4-78d48b357850" path="/var/lib/kubelet/pods/eede577b-8453-48ce-84d4-78d48b357850/volumes"
Jul 10 14:16:13.927601 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:13.927521    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:14.027434 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:14.027404    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:14.501956 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:14.501923    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:26.781140 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:26.781109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:27.031111 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.031080    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jtvnn"]
Jul 10 14:16:27.031425 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.031390    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eede577b-8453-48ce-84d4-78d48b357850" containerName="authorino"
Jul 10 14:16:27.031425 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.031403    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="eede577b-8453-48ce-84d4-78d48b357850" containerName="authorino"
Jul 10 14:16:27.031499 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.031467    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="eede577b-8453-48ce-84d4-78d48b357850" containerName="authorino"
Jul 10 14:16:27.033403 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.033382    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jtvnn"
Jul 10 14:16:27.043016 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.042995    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jtvnn"]
Jul 10 14:16:27.043181 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.043055    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:27.155032 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.154990    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bllnl\" (UniqueName: \"kubernetes.io/projected/d8991b56-155b-4bb4-8ab0-23185a2f93d1-kube-api-access-bllnl\") pod \"authorino-5d4b67c446-jtvnn\" (UID: \"d8991b56-155b-4bb4-8ab0-23185a2f93d1\") " pod="kuadrant-system/authorino-5d4b67c446-jtvnn"
Jul 10 14:16:27.255877 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.255843    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bllnl\" (UniqueName: \"kubernetes.io/projected/d8991b56-155b-4bb4-8ab0-23185a2f93d1-kube-api-access-bllnl\") pod \"authorino-5d4b67c446-jtvnn\" (UID: \"d8991b56-155b-4bb4-8ab0-23185a2f93d1\") " pod="kuadrant-system/authorino-5d4b67c446-jtvnn"
Jul 10 14:16:27.268781 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.268754    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bllnl\" (UniqueName: \"kubernetes.io/projected/d8991b56-155b-4bb4-8ab0-23185a2f93d1-kube-api-access-bllnl\") pod \"authorino-5d4b67c446-jtvnn\" (UID: \"d8991b56-155b-4bb4-8ab0-23185a2f93d1\") " pod="kuadrant-system/authorino-5d4b67c446-jtvnn"
Jul 10 14:16:27.304767 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.304683    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jtvnn"]
Jul 10 14:16:27.304948 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.304936    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jtvnn"
Jul 10 14:16:27.429518 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.429492    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jtvnn"]
Jul 10 14:16:27.431409 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:16:27.431382    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8991b56_155b_4bb4_8ab0_23185a2f93d1.slice/crio-a34533f12e6927669ceac8c2523d006f2223b27df0738cc8c783fd6abb47f003 WatchSource:0}: Error finding container a34533f12e6927669ceac8c2523d006f2223b27df0738cc8c783fd6abb47f003: Status 404 returned error can't find the container with id a34533f12e6927669ceac8c2523d006f2223b27df0738cc8c783fd6abb47f003
Jul 10 14:16:27.944960 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:27.944933    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jtvnn" event={"ID":"d8991b56-155b-4bb4-8ab0-23185a2f93d1","Type":"ContainerStarted","Data":"a34533f12e6927669ceac8c2523d006f2223b27df0738cc8c783fd6abb47f003"}
Jul 10 14:16:28.950570 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:28.950531    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jtvnn" event={"ID":"d8991b56-155b-4bb4-8ab0-23185a2f93d1","Type":"ContainerStarted","Data":"376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0"}
Jul 10 14:16:28.951037 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:28.950630    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jtvnn" podUID="d8991b56-155b-4bb4-8ab0-23185a2f93d1" containerName="authorino" containerID="cri-o://376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0" gracePeriod=30
Jul 10 14:16:28.967113 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:28.967067    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jtvnn" podStartSLOduration=1.505654507 podStartE2EDuration="1.967050692s" totalImagesPullingTime="461.396185ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:16:28.966310097 +0000 UTC m=+460.592036027" watchObservedRunningTime="2026-07-10 14:16:28.967050692 +0000 UTC m=+460.592776679"
Jul 10 14:16:29.195691 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.195659    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jtvnn"
Jul 10 14:16:29.272649 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.272564    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bllnl\" (UniqueName: \"kubernetes.io/projected/d8991b56-155b-4bb4-8ab0-23185a2f93d1-kube-api-access-bllnl\") pod \"d8991b56-155b-4bb4-8ab0-23185a2f93d1\" (UID: \"d8991b56-155b-4bb4-8ab0-23185a2f93d1\") "
Jul 10 14:16:29.274614 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.274583    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8991b56-155b-4bb4-8ab0-23185a2f93d1-kube-api-access-bllnl" (OuterVolumeSpecName: "kube-api-access-bllnl") pod "d8991b56-155b-4bb4-8ab0-23185a2f93d1" (UID: "d8991b56-155b-4bb4-8ab0-23185a2f93d1"). InnerVolumeSpecName "kube-api-access-bllnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:16:29.374010 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.373977    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bllnl\" (UniqueName: \"kubernetes.io/projected/d8991b56-155b-4bb4-8ab0-23185a2f93d1-kube-api-access-bllnl\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:16:29.956116 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.956074    2584 generic.go:356] "Generic (PLEG): container finished" podID="d8991b56-155b-4bb4-8ab0-23185a2f93d1" containerID="376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0" exitCode=0
Jul 10 14:16:29.956514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.956130    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jtvnn"
Jul 10 14:16:29.956514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.956141    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jtvnn" event={"ID":"d8991b56-155b-4bb4-8ab0-23185a2f93d1","Type":"ContainerDied","Data":"376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0"}
Jul 10 14:16:29.956514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.956181    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jtvnn" event={"ID":"d8991b56-155b-4bb4-8ab0-23185a2f93d1","Type":"ContainerDied","Data":"a34533f12e6927669ceac8c2523d006f2223b27df0738cc8c783fd6abb47f003"}
Jul 10 14:16:29.956514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.956202    2584 scope.go:117] "RemoveContainer" containerID="376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0"
Jul 10 14:16:29.964820 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.964803    2584 scope.go:117] "RemoveContainer" containerID="376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0"
Jul 10 14:16:29.965081 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:16:29.965066    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0\": container with ID starting with 376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0 not found: ID does not exist" containerID="376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0"
Jul 10 14:16:29.965119 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.965088    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0"} err="failed to get container status \"376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0\": rpc error: code = NotFound desc = could not find container \"376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0\": container with ID starting with 376926948ea29780ce821b47909115fedd7e9199516296ee092cd44e606e83d0 not found: ID does not exist"
Jul 10 14:16:29.978555 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.978533    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jtvnn"]
Jul 10 14:16:29.982566 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:29.982545    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jtvnn"]
Jul 10 14:16:30.920369 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:30.920336    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8991b56-155b-4bb4-8ab0-23185a2f93d1" path="/var/lib/kubelet/pods/d8991b56-155b-4bb4-8ab0-23185a2f93d1/volumes"
Jul 10 14:16:32.001968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.001934    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7rdzh"]
Jul 10 14:16:32.002347 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.002153    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7rdzh" podUID="9b05f767-323d-47db-b0bd-16baf6b16ef2" containerName="authorino" containerID="cri-o://91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b" gracePeriod=30
Jul 10 14:16:32.266704 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.266637    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7rdzh"
Jul 10 14:16:32.402363 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.402334    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6zsj\" (UniqueName: \"kubernetes.io/projected/9b05f767-323d-47db-b0bd-16baf6b16ef2-kube-api-access-t6zsj\") pod \"9b05f767-323d-47db-b0bd-16baf6b16ef2\" (UID: \"9b05f767-323d-47db-b0bd-16baf6b16ef2\") "
Jul 10 14:16:32.405884 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.405850    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b05f767-323d-47db-b0bd-16baf6b16ef2-kube-api-access-t6zsj" (OuterVolumeSpecName: "kube-api-access-t6zsj") pod "9b05f767-323d-47db-b0bd-16baf6b16ef2" (UID: "9b05f767-323d-47db-b0bd-16baf6b16ef2"). InnerVolumeSpecName "kube-api-access-t6zsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:16:32.503098 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.503058    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6zsj\" (UniqueName: \"kubernetes.io/projected/9b05f767-323d-47db-b0bd-16baf6b16ef2-kube-api-access-t6zsj\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:16:32.970559 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.970468    2584 generic.go:356] "Generic (PLEG): container finished" podID="9b05f767-323d-47db-b0bd-16baf6b16ef2" containerID="91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b" exitCode=0
Jul 10 14:16:32.970559 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.970512    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7rdzh"
Jul 10 14:16:32.970559 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.970523    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7rdzh" event={"ID":"9b05f767-323d-47db-b0bd-16baf6b16ef2","Type":"ContainerDied","Data":"91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b"}
Jul 10 14:16:32.970559 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.970564    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7rdzh" event={"ID":"9b05f767-323d-47db-b0bd-16baf6b16ef2","Type":"ContainerDied","Data":"eaeca54aaaace04ba7d29402e3a77922e238dcd3340df1b91826e04d54fdb9e9"}
Jul 10 14:16:32.970906 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.970581    2584 scope.go:117] "RemoveContainer" containerID="91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b"
Jul 10 14:16:32.978503 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.978486    2584 scope.go:117] "RemoveContainer" containerID="91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b"
Jul 10 14:16:32.978784 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:16:32.978766    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b\": container with ID starting with 91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b not found: ID does not exist" containerID="91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b"
Jul 10 14:16:32.978833 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.978793    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b"} err="failed to get container status \"91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b\": rpc error: code = NotFound desc = could not find container \"91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b\": container with ID starting with 91b7095fdd46ba83b42bf29f81e8bed2b423cf428938d20d78103a0d45898a9b not found: ID does not exist"
Jul 10 14:16:32.989113 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.989085    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7rdzh"]
Jul 10 14:16:32.993205 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:32.993183    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7rdzh"]
Jul 10 14:16:34.921040 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:34.921008    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b05f767-323d-47db-b0bd-16baf6b16ef2" path="/var/lib/kubelet/pods/9b05f767-323d-47db-b0bd-16baf6b16ef2/volumes"
Jul 10 14:16:36.494517 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.494479    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:36.520230 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.520190    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67cdc7578c-gw5j2"]
Jul 10 14:16:36.520566 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.520549    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b05f767-323d-47db-b0bd-16baf6b16ef2" containerName="authorino"
Jul 10 14:16:36.520644 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.520569    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b05f767-323d-47db-b0bd-16baf6b16ef2" containerName="authorino"
Jul 10 14:16:36.520644 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.520584    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8991b56-155b-4bb4-8ab0-23185a2f93d1" containerName="authorino"
Jul 10 14:16:36.520644 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.520592    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8991b56-155b-4bb4-8ab0-23185a2f93d1" containerName="authorino"
Jul 10 14:16:36.520824 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.520683    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b05f767-323d-47db-b0bd-16baf6b16ef2" containerName="authorino"
Jul 10 14:16:36.520824 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.520699    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8991b56-155b-4bb4-8ab0-23185a2f93d1" containerName="authorino"
Jul 10 14:16:36.523589 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.523568    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:36.527623 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.527598    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 14:16:36.528465 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.528447    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6dh45\""
Jul 10 14:16:36.529053 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.529035    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 14:16:36.539822 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.539793    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-gw5j2"]
Jul 10 14:16:36.593474 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.593430    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:36.639538 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.639501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert\") pod \"maas-controller-67cdc7578c-gw5j2\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") " pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:36.639708 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.639549    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv4cb\" (UniqueName: \"kubernetes.io/projected/46635175-e1cb-48cc-aa7b-77b154dac2ae-kube-api-access-nv4cb\") pod \"maas-controller-67cdc7578c-gw5j2\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") " pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:36.660715 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.660679    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:36.740957 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.740919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert\") pod \"maas-controller-67cdc7578c-gw5j2\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") " pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:36.740957 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.740961    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv4cb\" (UniqueName: \"kubernetes.io/projected/46635175-e1cb-48cc-aa7b-77b154dac2ae-kube-api-access-nv4cb\") pod \"maas-controller-67cdc7578c-gw5j2\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") " pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:36.741199 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:16:36.741084    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 10 14:16:36.741199 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:16:36.741160    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert podName:46635175-e1cb-48cc-aa7b-77b154dac2ae nodeName:}" failed. No retries permitted until 2026-07-10 14:16:37.241137838 +0000 UTC m=+468.866863754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert") pod "maas-controller-67cdc7578c-gw5j2" (UID: "46635175-e1cb-48cc-aa7b-77b154dac2ae") : secret "maas-controller-webhook-cert" not found
Jul 10 14:16:36.750232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.750171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv4cb\" (UniqueName: \"kubernetes.io/projected/46635175-e1cb-48cc-aa7b-77b154dac2ae-kube-api-access-nv4cb\") pod \"maas-controller-67cdc7578c-gw5j2\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") " pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:36.831645 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:36.831612    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:37.246098 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:37.246055    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert\") pod \"maas-controller-67cdc7578c-gw5j2\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") " pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:37.248460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:37.248431    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert\") pod \"maas-controller-67cdc7578c-gw5j2\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") " pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:37.436292 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:37.436253    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:37.567236 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:37.567113    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-gw5j2"]
Jul 10 14:16:37.569494 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:16:37.569466    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46635175_e1cb_48cc_aa7b_77b154dac2ae.slice/crio-735e5b8531b1337390b4c6a66334d216a8f02ad0712aa8ea0e9195cf43599781 WatchSource:0}: Error finding container 735e5b8531b1337390b4c6a66334d216a8f02ad0712aa8ea0e9195cf43599781: Status 404 returned error can't find the container with id 735e5b8531b1337390b4c6a66334d216a8f02ad0712aa8ea0e9195cf43599781
Jul 10 14:16:37.993507 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:37.993476    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-gw5j2" event={"ID":"46635175-e1cb-48cc-aa7b-77b154dac2ae","Type":"ContainerStarted","Data":"735e5b8531b1337390b4c6a66334d216a8f02ad0712aa8ea0e9195cf43599781"}
Jul 10 14:16:39.504496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:39.504435    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:39.608146 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:39.608118    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:40.004644 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:40.004606    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-gw5j2" event={"ID":"46635175-e1cb-48cc-aa7b-77b154dac2ae","Type":"ContainerStarted","Data":"540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14"}
Jul 10 14:16:40.004813 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:40.004669    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:40.035056 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:40.035003    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67cdc7578c-gw5j2" podStartSLOduration=1.963250481 podStartE2EDuration="4.034981051s" totalImagesPullingTime="2.07173057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:16:40.033943395 +0000 UTC m=+471.659669327" watchObservedRunningTime="2026-07-10 14:16:40.034981051 +0000 UTC m=+471.660706983"
Jul 10 14:16:51.014743 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.014680    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:51.069387 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.069352    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-gw5j2"]
Jul 10 14:16:51.069595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.069549    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67cdc7578c-gw5j2" podUID="46635175-e1cb-48cc-aa7b-77b154dac2ae" containerName="manager" containerID="cri-o://540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14" gracePeriod=30
Jul 10 14:16:51.309275 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.309252    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:51.473706 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.473668    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv4cb\" (UniqueName: \"kubernetes.io/projected/46635175-e1cb-48cc-aa7b-77b154dac2ae-kube-api-access-nv4cb\") pod \"46635175-e1cb-48cc-aa7b-77b154dac2ae\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") "
Jul 10 14:16:51.473903 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.473751    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert\") pod \"46635175-e1cb-48cc-aa7b-77b154dac2ae\" (UID: \"46635175-e1cb-48cc-aa7b-77b154dac2ae\") "
Jul 10 14:16:51.475899 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.475860    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "46635175-e1cb-48cc-aa7b-77b154dac2ae" (UID: "46635175-e1cb-48cc-aa7b-77b154dac2ae"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:16:51.476014 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.475903    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46635175-e1cb-48cc-aa7b-77b154dac2ae-kube-api-access-nv4cb" (OuterVolumeSpecName: "kube-api-access-nv4cb") pod "46635175-e1cb-48cc-aa7b-77b154dac2ae" (UID: "46635175-e1cb-48cc-aa7b-77b154dac2ae"). InnerVolumeSpecName "kube-api-access-nv4cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:16:51.574496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.574464    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv4cb\" (UniqueName: \"kubernetes.io/projected/46635175-e1cb-48cc-aa7b-77b154dac2ae-kube-api-access-nv4cb\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:16:51.574496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:51.574495    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46635175-e1cb-48cc-aa7b-77b154dac2ae-webhook-cert\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:16:52.058886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.058801    2584 generic.go:356] "Generic (PLEG): container finished" podID="46635175-e1cb-48cc-aa7b-77b154dac2ae" containerID="540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14" exitCode=0
Jul 10 14:16:52.058886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.058852    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67cdc7578c-gw5j2"
Jul 10 14:16:52.058886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.058859    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-gw5j2" event={"ID":"46635175-e1cb-48cc-aa7b-77b154dac2ae","Type":"ContainerDied","Data":"540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14"}
Jul 10 14:16:52.059349 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.058903    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67cdc7578c-gw5j2" event={"ID":"46635175-e1cb-48cc-aa7b-77b154dac2ae","Type":"ContainerDied","Data":"735e5b8531b1337390b4c6a66334d216a8f02ad0712aa8ea0e9195cf43599781"}
Jul 10 14:16:52.059349 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.058924    2584 scope.go:117] "RemoveContainer" containerID="540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14"
Jul 10 14:16:52.068015 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.067998    2584 scope.go:117] "RemoveContainer" containerID="540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14"
Jul 10 14:16:52.068305 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:16:52.068277    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14\": container with ID starting with 540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14 not found: ID does not exist" containerID="540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14"
Jul 10 14:16:52.068400 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.068310    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14"} err="failed to get container status \"540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14\": rpc error: code = NotFound desc = could not find container \"540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14\": container with ID starting with 540e03bd9dd8bf8837843f71d28b87a441d6c11b78ea19b09c7847ad233abf14 not found: ID does not exist"
Jul 10 14:16:52.085167 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.085134    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-gw5j2"]
Jul 10 14:16:52.089885 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.089859    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67cdc7578c-gw5j2"]
Jul 10 14:16:52.741638 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.741606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:16:52.920211 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:52.920180    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46635175-e1cb-48cc-aa7b-77b154dac2ae" path="/var/lib/kubelet/pods/46635175-e1cb-48cc-aa7b-77b154dac2ae/volumes"
Jul 10 14:16:53.049260 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:16:53.049182    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:09.683754 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:09.683711    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:10.177842 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.177812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:10.228567 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.228536    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"]
Jul 10 14:17:10.228890 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.228875    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46635175-e1cb-48cc-aa7b-77b154dac2ae" containerName="manager"
Jul 10 14:17:10.228953 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.228893    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="46635175-e1cb-48cc-aa7b-77b154dac2ae" containerName="manager"
Jul 10 14:17:10.228989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.228962    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="46635175-e1cb-48cc-aa7b-77b154dac2ae" containerName="manager"
Jul 10 14:17:10.231353 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.231339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.234283 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.234261    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 14:17:10.234384 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.234353    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 10 14:17:10.235485 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.235469    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzvlc\""
Jul 10 14:17:10.235543 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.235482    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 14:17:10.242636 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.242616    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"]
Jul 10 14:17:10.322294 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.322264    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:10.329435 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.329415    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03d95c8-6cb7-4b14-8045-52afa8679b5d-openshift-service-ca\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.329498 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.329445    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03d95c8-6cb7-4b14-8045-52afa8679b5d-maas-api-tls\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.329498 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.329484    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vqlt\" (UniqueName: \"kubernetes.io/projected/a03d95c8-6cb7-4b14-8045-52afa8679b5d-kube-api-access-6vqlt\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.430880 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.430788    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03d95c8-6cb7-4b14-8045-52afa8679b5d-openshift-service-ca\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.430880 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.430838    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03d95c8-6cb7-4b14-8045-52afa8679b5d-maas-api-tls\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.431059 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.430890    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vqlt\" (UniqueName: \"kubernetes.io/projected/a03d95c8-6cb7-4b14-8045-52afa8679b5d-kube-api-access-6vqlt\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.431429 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.431400    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03d95c8-6cb7-4b14-8045-52afa8679b5d-openshift-service-ca\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.433215 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.433196    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03d95c8-6cb7-4b14-8045-52afa8679b5d-maas-api-tls\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.441162 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.441141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vqlt\" (UniqueName: \"kubernetes.io/projected/a03d95c8-6cb7-4b14-8045-52afa8679b5d-kube-api-access-6vqlt\") pod \"maas-api-68fc5bb9d9-6xdsf\" (UID: \"a03d95c8-6cb7-4b14-8045-52afa8679b5d\") " pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.540812 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.540776    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:10.679444 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.679420    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"]
Jul 10 14:17:10.681890 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:17:10.681825    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03d95c8_6cb7_4b14_8045_52afa8679b5d.slice/crio-5db45dbadd7b8f580316e35c9a97b575d848caa7ecaac81623183a2d70ba48ce WatchSource:0}: Error finding container 5db45dbadd7b8f580316e35c9a97b575d848caa7ecaac81623183a2d70ba48ce: Status 404 returned error can't find the container with id 5db45dbadd7b8f580316e35c9a97b575d848caa7ecaac81623183a2d70ba48ce
Jul 10 14:17:10.741104 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.741053    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:10.841556 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.841524    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:10.872088 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.872054    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cch5m"]
Jul 10 14:17:10.875303 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.875280    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:10.878042 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.878023    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 10 14:17:10.878163 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.878087    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dlfml\""
Jul 10 14:17:10.886249 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.886228    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cch5m"]
Jul 10 14:17:10.941704 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.941615    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flgxn\" (UniqueName: \"kubernetes.io/projected/8f9190d8-cc06-4879-a490-56a43de21be8-kube-api-access-flgxn\") pod \"payload-processing-58f86c7798-cch5m\" (UID: \"8f9190d8-cc06-4879-a490-56a43de21be8\") " pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:10.941704 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:10.941664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f9190d8-cc06-4879-a490-56a43de21be8-plugins-config-volume\") pod \"payload-processing-58f86c7798-cch5m\" (UID: \"8f9190d8-cc06-4879-a490-56a43de21be8\") " pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:11.042587 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:11.042554    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flgxn\" (UniqueName: \"kubernetes.io/projected/8f9190d8-cc06-4879-a490-56a43de21be8-kube-api-access-flgxn\") pod \"payload-processing-58f86c7798-cch5m\" (UID: \"8f9190d8-cc06-4879-a490-56a43de21be8\") " pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:11.042760 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:11.042631    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f9190d8-cc06-4879-a490-56a43de21be8-plugins-config-volume\") pod \"payload-processing-58f86c7798-cch5m\" (UID: \"8f9190d8-cc06-4879-a490-56a43de21be8\") " pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:11.043377 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:11.043353    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f9190d8-cc06-4879-a490-56a43de21be8-plugins-config-volume\") pod \"payload-processing-58f86c7798-cch5m\" (UID: \"8f9190d8-cc06-4879-a490-56a43de21be8\") " pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:11.052833 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:11.052810    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flgxn\" (UniqueName: \"kubernetes.io/projected/8f9190d8-cc06-4879-a490-56a43de21be8-kube-api-access-flgxn\") pod \"payload-processing-58f86c7798-cch5m\" (UID: \"8f9190d8-cc06-4879-a490-56a43de21be8\") " pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:11.143735 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:11.143693    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf" event={"ID":"a03d95c8-6cb7-4b14-8045-52afa8679b5d","Type":"ContainerStarted","Data":"5db45dbadd7b8f580316e35c9a97b575d848caa7ecaac81623183a2d70ba48ce"}
Jul 10 14:17:11.186993 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:11.186961    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:11.313069 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:11.312976    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-cch5m"]
Jul 10 14:17:11.315679 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:17:11.315637    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9190d8_cc06_4879_a490_56a43de21be8.slice/crio-d21cf370d3270e10a0c149567ef043afdfaf0cd74b801aab61312ae2f1cb7919 WatchSource:0}: Error finding container d21cf370d3270e10a0c149567ef043afdfaf0cd74b801aab61312ae2f1cb7919: Status 404 returned error can't find the container with id d21cf370d3270e10a0c149567ef043afdfaf0cd74b801aab61312ae2f1cb7919
Jul 10 14:17:12.150291 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:12.150252    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-cch5m" event={"ID":"8f9190d8-cc06-4879-a490-56a43de21be8","Type":"ContainerStarted","Data":"d21cf370d3270e10a0c149567ef043afdfaf0cd74b801aab61312ae2f1cb7919"}
Jul 10 14:17:13.156562 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:13.156526    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-cch5m" event={"ID":"8f9190d8-cc06-4879-a490-56a43de21be8","Type":"ContainerStarted","Data":"93627d1c6f58a396b25a293e917a5f2921f0e91c0f4a857e8eb7204ea468d8f4"}
Jul 10 14:17:13.156913 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:13.156635    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:13.174229 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:13.174182    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-cch5m" podStartSLOduration=1.785539035 podStartE2EDuration="3.174163687s" totalImagesPullingTime="1.388624652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:17:13.172699103 +0000 UTC m=+504.798425036" watchObservedRunningTime="2026-07-10 14:17:13.174163687 +0000 UTC m=+504.799889643"
Jul 10 14:17:14.162522 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:14.162483    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf" event={"ID":"a03d95c8-6cb7-4b14-8045-52afa8679b5d","Type":"ContainerStarted","Data":"f8c06f318fafd2aaf77bcb5f3c4f2304ab801d03952aa572ce40103e92d3b3b6"}
Jul 10 14:17:14.163003 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:14.162637    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:14.181776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:14.181704    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf" podStartSLOduration=1.7078659200000001 podStartE2EDuration="4.181689124s" totalImagesPullingTime="2.473823204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:17:14.1811156 +0000 UTC m=+505.806841523" watchObservedRunningTime="2026-07-10 14:17:14.181689124 +0000 UTC m=+505.807415053"
Jul 10 14:17:20.171827 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:20.171797    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68fc5bb9d9-6xdsf"
Jul 10 14:17:21.169591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:21.169557    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:21.269232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:21.269197    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:24.164782 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:24.164750    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-cch5m"
Jul 10 14:17:29.095593 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:29.095551    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:37.485645 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:37.485613    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:52.355949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.355909    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"]
Jul 10 14:17:52.360549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.360516    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.366969 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.364964    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 14:17:52.366969 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.365235    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 10 14:17:52.366969 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.365463    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8rsbh\""
Jul 10 14:17:52.366969 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.365703    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 14:17:52.371282 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.371254    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"]
Jul 10 14:17:52.409180 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.409147    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94zw4\" (UniqueName: \"kubernetes.io/projected/25e6d4b8-11ae-464a-9523-5ad58c45694d-kube-api-access-94zw4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.409365 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.409202    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.409365 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.409277    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.409365 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.409324    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.409365 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.409343    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25e6d4b8-11ae-464a-9523-5ad58c45694d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.409365 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.409361    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.454630 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.454593    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:52.509859 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.509818    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510049 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.509872    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510049 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.510035    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25e6d4b8-11ae-464a-9523-5ad58c45694d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510163 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.510069    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510163 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.510102    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94zw4\" (UniqueName: \"kubernetes.io/projected/25e6d4b8-11ae-464a-9523-5ad58c45694d-kube-api-access-94zw4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510163 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.510150    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.510240    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.510302    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.510396 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.510380    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.512357 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.512327    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25e6d4b8-11ae-464a-9523-5ad58c45694d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.512504 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.512487    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25e6d4b8-11ae-464a-9523-5ad58c45694d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.519496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.519472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94zw4\" (UniqueName: \"kubernetes.io/projected/25e6d4b8-11ae-464a-9523-5ad58c45694d-kube-api-access-94zw4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rvk86\" (UID: \"25e6d4b8-11ae-464a-9523-5ad58c45694d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.674853 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.674764    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:17:52.809538 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:52.809515    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"]
Jul 10 14:17:52.811475 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:17:52.811451    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25e6d4b8_11ae_464a_9523_5ad58c45694d.slice/crio-a9aba154e65138642f3b685228f7dbe34870eb1582573c2ab77ec09d82e81b3b WatchSource:0}: Error finding container a9aba154e65138642f3b685228f7dbe34870eb1582573c2ab77ec09d82e81b3b: Status 404 returned error can't find the container with id a9aba154e65138642f3b685228f7dbe34870eb1582573c2ab77ec09d82e81b3b
Jul 10 14:17:53.333532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:53.333490    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86" event={"ID":"25e6d4b8-11ae-464a-9523-5ad58c45694d","Type":"ContainerStarted","Data":"a9aba154e65138642f3b685228f7dbe34870eb1582573c2ab77ec09d82e81b3b"}
Jul 10 14:17:54.339970 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:54.339936    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86" event={"ID":"25e6d4b8-11ae-464a-9523-5ad58c45694d","Type":"ContainerStarted","Data":"a0b6d8fe5c23e4f5a93b3e277b948b6890158f97413a0f76d7ed4fa0c378cb3d"}
Jul 10 14:17:54.372799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:54.372748    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86" podStartSLOduration=1.298911124 podStartE2EDuration="2.372713004s" totalImagesPullingTime="1.07380188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:17:54.372243219 +0000 UTC m=+545.997969149" watchObservedRunningTime="2026-07-10 14:17:54.372713004 +0000 UTC m=+545.998438968"
Jul 10 14:17:56.603548 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:56.603511    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:17:59.346680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:17:59.346644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:18:00.982480 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:00.982437    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:18:02.675548 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:02.675512    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:18:02.676026 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:02.675809    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:18:02.688114 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:02.688087    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:18:03.390748 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:03.390704    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rvk86"
Jul 10 14:18:06.795265 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:06.795231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:18:15.852098 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:15.852064    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"]
Jul 10 14:18:15.854989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:15.854970    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:15.858289 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:15.858264    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 10 14:18:15.865124 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:15.865100    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"]
Jul 10 14:18:15.891164 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:15.891130    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:18:16.015954 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.015911    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.015954 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.015960    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.016207 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.016162    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.016272 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.016208    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.016272 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.016246    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.016350 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.016277    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tl8n\" (UniqueName: \"kubernetes.io/projected/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-kube-api-access-4tl8n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117320 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117237    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117320 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117288    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117528 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117322    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117528 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117349    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tl8n\" (UniqueName: \"kubernetes.io/projected/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-kube-api-access-4tl8n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117528 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117393    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117528 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117410    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117807 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117776    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117807 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117792    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.117958 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.117830    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.119483 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.119462    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.119765 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.119747    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.126195 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.126176    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tl8n\" (UniqueName: \"kubernetes.io/projected/39d7dfaf-3c69-4e0e-a244-e0ba3789296b-kube-api-access-4tl8n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg\" (UID: \"39d7dfaf-3c69-4e0e-a244-e0ba3789296b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.166373 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.166341    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:16.295142 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.295119    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"]
Jul 10 14:18:16.296848 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:18:16.296822    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39d7dfaf_3c69_4e0e_a244_e0ba3789296b.slice/crio-a9e2706844a8f9b4289844b5ce0f55790aa29b6f8cc504b011ea44929f6afd62 WatchSource:0}: Error finding container a9e2706844a8f9b4289844b5ce0f55790aa29b6f8cc504b011ea44929f6afd62: Status 404 returned error can't find the container with id a9e2706844a8f9b4289844b5ce0f55790aa29b6f8cc504b011ea44929f6afd62
Jul 10 14:18:16.439843 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:16.439764    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg" event={"ID":"39d7dfaf-3c69-4e0e-a244-e0ba3789296b","Type":"ContainerStarted","Data":"a9e2706844a8f9b4289844b5ce0f55790aa29b6f8cc504b011ea44929f6afd62"}
Jul 10 14:18:17.446164 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:17.446126    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg" event={"ID":"39d7dfaf-3c69-4e0e-a244-e0ba3789296b","Type":"ContainerStarted","Data":"f45b01105fc862118824d2f7d94a5ac26f41b319b3e970659752d0a201548ca6"}
Jul 10 14:18:17.465824 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:17.465767    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg" podStartSLOduration=2.259181206 podStartE2EDuration="2.465750159s" totalImagesPullingTime="206.568953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:18:17.464550406 +0000 UTC m=+569.090276338" watchObservedRunningTime="2026-07-10 14:18:17.465750159 +0000 UTC m=+569.091476091"
Jul 10 14:18:26.167043 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:26.167006    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:26.167503 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:26.167058    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:26.179336 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:26.179315    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:26.495977 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:26.495891    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg"
Jul 10 14:18:48.778445 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:48.778415    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:18:48.782006 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:18:48.781987    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:19:05.744564 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.744527    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dcbb7dc9c-ph7jd"]
Jul 10 14:19:05.747001 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.746980    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:05.751190 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.751172    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 14:19:05.751280 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.751176    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cmfx9\""
Jul 10 14:19:05.755345 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.755312    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dcbb7dc9c-ph7jd"]
Jul 10 14:19:05.843612 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.843575    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljgs9\" (UniqueName: \"kubernetes.io/projected/267e171c-f86e-482d-806c-736b0ab32fd7-kube-api-access-ljgs9\") pod \"authorino-6dcbb7dc9c-ph7jd\" (UID: \"267e171c-f86e-482d-806c-736b0ab32fd7\") " pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:05.843814 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.843632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/267e171c-f86e-482d-806c-736b0ab32fd7-tls-cert\") pod \"authorino-6dcbb7dc9c-ph7jd\" (UID: \"267e171c-f86e-482d-806c-736b0ab32fd7\") " pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:05.944228 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.944189    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljgs9\" (UniqueName: \"kubernetes.io/projected/267e171c-f86e-482d-806c-736b0ab32fd7-kube-api-access-ljgs9\") pod \"authorino-6dcbb7dc9c-ph7jd\" (UID: \"267e171c-f86e-482d-806c-736b0ab32fd7\") " pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:05.944386 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.944258    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/267e171c-f86e-482d-806c-736b0ab32fd7-tls-cert\") pod \"authorino-6dcbb7dc9c-ph7jd\" (UID: \"267e171c-f86e-482d-806c-736b0ab32fd7\") " pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:05.946643 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.946624    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/267e171c-f86e-482d-806c-736b0ab32fd7-tls-cert\") pod \"authorino-6dcbb7dc9c-ph7jd\" (UID: \"267e171c-f86e-482d-806c-736b0ab32fd7\") " pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:05.952162 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:05.952145    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljgs9\" (UniqueName: \"kubernetes.io/projected/267e171c-f86e-482d-806c-736b0ab32fd7-kube-api-access-ljgs9\") pod \"authorino-6dcbb7dc9c-ph7jd\" (UID: \"267e171c-f86e-482d-806c-736b0ab32fd7\") " pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:06.057293 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:06.057205    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd"
Jul 10 14:19:06.183137 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:06.183018    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dcbb7dc9c-ph7jd"]
Jul 10 14:19:06.185319 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:19:06.185293    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod267e171c_f86e_482d_806c_736b0ab32fd7.slice/crio-10d089e00bfa3d3f24f9ca3797e660c2989a6c68027e30781de8fc290338ab92 WatchSource:0}: Error finding container 10d089e00bfa3d3f24f9ca3797e660c2989a6c68027e30781de8fc290338ab92: Status 404 returned error can't find the container with id 10d089e00bfa3d3f24f9ca3797e660c2989a6c68027e30781de8fc290338ab92
Jul 10 14:19:06.661451 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:06.661407    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd" event={"ID":"267e171c-f86e-482d-806c-736b0ab32fd7","Type":"ContainerStarted","Data":"10d089e00bfa3d3f24f9ca3797e660c2989a6c68027e30781de8fc290338ab92"}
Jul 10 14:19:07.669200 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:07.669166    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd" event={"ID":"267e171c-f86e-482d-806c-736b0ab32fd7","Type":"ContainerStarted","Data":"f7eda787455faffe07c6fec441e7504e1c92217b1f91ddfbb7f0a16efee094e3"}
Jul 10 14:19:07.693828 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:07.693779    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dcbb7dc9c-ph7jd" podStartSLOduration=2.174739827 podStartE2EDuration="2.693762183s" totalImagesPullingTime="519.022356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:19:07.692555769 +0000 UTC m=+619.318281704" watchObservedRunningTime="2026-07-10 14:19:07.693762183 +0000 UTC m=+619.319488114"
Jul 10 14:19:08.276629 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:08.276595    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:19:08.372145 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:08.372108    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:19:09.454680 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:09.454644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:19:09.556519 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:09.556492    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:19:13.566586 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:19:13.566515    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:20:30.998397 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:20:30.998361    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:20:32.932316 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:20:32.932283    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:20:46.618349 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:20:46.618314    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:20:46.676283 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:20:46.676255    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:06.983667 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:06.983631    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:07.079564 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:07.079530    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:07.219935 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:07.219900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:14.080527 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:14.080487    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:14.220415 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:14.220385    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:14.443414 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:14.443328    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:14.580134 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:14.580095    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:21.425630 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:21.425590    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:21:21.540519 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:21:21.540482    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:22:36.981594 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:22:36.981514    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:23:34.717326 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:23:34.717291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:23:48.806630 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:23:48.806548    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:23:48.808165 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:23:48.808145    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:25:16.633228 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:16.633148    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"]
Jul 10 14:25:16.635749 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:16.633395    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2" podUID="a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" containerName="manager" containerID="cri-o://d74610a4c753d21744aa0cecb9e40277879070b1b437fe9a3146d5e12ed4206c" gracePeriod=10
Jul 10 14:25:20.335196 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.335158    2584 generic.go:356] "Generic (PLEG): container finished" podID="a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" containerID="d74610a4c753d21744aa0cecb9e40277879070b1b437fe9a3146d5e12ed4206c" exitCode=0
Jul 10 14:25:20.335706 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.335232    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2" event={"ID":"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb","Type":"ContainerDied","Data":"d74610a4c753d21744aa0cecb9e40277879070b1b437fe9a3146d5e12ed4206c"}
Jul 10 14:25:20.579573 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.579547    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:25:20.712434 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.712349    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vjn8\" (UniqueName: \"kubernetes.io/projected/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-kube-api-access-8vjn8\") pod \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") "
Jul 10 14:25:20.712583 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.712461    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-extensions-socket-volume\") pod \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\" (UID: \"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb\") "
Jul 10 14:25:20.712887 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.712864    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" (UID: "a6f8214c-150a-4fc2-8e0d-6080ffd1cacb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:25:20.714558 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.714536    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-kube-api-access-8vjn8" (OuterVolumeSpecName: "kube-api-access-8vjn8") pod "a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" (UID: "a6f8214c-150a-4fc2-8e0d-6080ffd1cacb"). InnerVolumeSpecName "kube-api-access-8vjn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:25:20.813409 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.813374    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vjn8\" (UniqueName: \"kubernetes.io/projected/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-kube-api-access-8vjn8\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:25:20.813409 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:20.813403    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb-extensions-socket-volume\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:25:21.341058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:21.341033    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"
Jul 10 14:25:21.341058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:21.341043    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2" event={"ID":"a6f8214c-150a-4fc2-8e0d-6080ffd1cacb","Type":"ContainerDied","Data":"6dfaf6a61602b3ca4da37e678920d2061b93784323b4d5eee9b935fc4da109f3"}
Jul 10 14:25:21.341578 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:21.341106    2584 scope.go:117] "RemoveContainer" containerID="d74610a4c753d21744aa0cecb9e40277879070b1b437fe9a3146d5e12ed4206c"
Jul 10 14:25:21.361803 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:21.361780    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"]
Jul 10 14:25:21.367745 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:21.366774    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g87r2"]
Jul 10 14:25:22.920591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:25:22.920558    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" path="/var/lib/kubelet/pods/a6f8214c-150a-4fc2-8e0d-6080ffd1cacb/volumes"
Jul 10 14:26:17.813344 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.813311    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"]
Jul 10 14:26:17.813932 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.813787    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" containerName="manager"
Jul 10 14:26:17.813932 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.813806    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" containerName="manager"
Jul 10 14:26:17.813932 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.813923    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6f8214c-150a-4fc2-8e0d-6080ffd1cacb" containerName="manager"
Jul 10 14:26:17.816965 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.816944    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:17.821079 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.821060    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lxhdp\""
Jul 10 14:26:17.841837 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.841817    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"]
Jul 10 14:26:17.859994 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.859970    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f35817c-d7bb-48dc-a478-a2cdd8bf2d66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p8pxd\" (UID: \"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:17.860123 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.860001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdsbl\" (UniqueName: \"kubernetes.io/projected/4f35817c-d7bb-48dc-a478-a2cdd8bf2d66-kube-api-access-wdsbl\") pod \"kuadrant-operator-controller-manager-86878c769f-p8pxd\" (UID: \"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:17.960553 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.960525    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f35817c-d7bb-48dc-a478-a2cdd8bf2d66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p8pxd\" (UID: \"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:17.960553 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.960557    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdsbl\" (UniqueName: \"kubernetes.io/projected/4f35817c-d7bb-48dc-a478-a2cdd8bf2d66-kube-api-access-wdsbl\") pod \"kuadrant-operator-controller-manager-86878c769f-p8pxd\" (UID: \"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:17.960938 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.960919    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f35817c-d7bb-48dc-a478-a2cdd8bf2d66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p8pxd\" (UID: \"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:17.970777 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:17.970750    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdsbl\" (UniqueName: \"kubernetes.io/projected/4f35817c-d7bb-48dc-a478-a2cdd8bf2d66-kube-api-access-wdsbl\") pod \"kuadrant-operator-controller-manager-86878c769f-p8pxd\" (UID: \"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:18.127332 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:18.127306    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:18.281781 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:18.281751    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"]
Jul 10 14:26:18.283422 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:26:18.283388    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f35817c_d7bb_48dc_a478_a2cdd8bf2d66.slice/crio-26369df3820f9da014311aae10f2f839de051224a928c2cda9dfb3cc34a52573 WatchSource:0}: Error finding container 26369df3820f9da014311aae10f2f839de051224a928c2cda9dfb3cc34a52573: Status 404 returned error can't find the container with id 26369df3820f9da014311aae10f2f839de051224a928c2cda9dfb3cc34a52573
Jul 10 14:26:18.605844 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:18.605812    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd" event={"ID":"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66","Type":"ContainerStarted","Data":"64e014b6212186b424806ee7ab219e4e402ae999fb93b0f6ca4f96af33386a7e"}
Jul 10 14:26:18.606198 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:18.605861    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd" event={"ID":"4f35817c-d7bb-48dc-a478-a2cdd8bf2d66","Type":"ContainerStarted","Data":"26369df3820f9da014311aae10f2f839de051224a928c2cda9dfb3cc34a52573"}
Jul 10 14:26:18.606198 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:18.605891    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:18.632172 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:18.632120    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd" podStartSLOduration=1.632104668 podStartE2EDuration="1.632104668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:26:18.6311902 +0000 UTC m=+1050.256916150" watchObservedRunningTime="2026-07-10 14:26:18.632104668 +0000 UTC m=+1050.257830599"
Jul 10 14:26:29.613420 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:29.613388    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p8pxd"
Jul 10 14:26:37.916348 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:26:37.916311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:28:23.041057 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:28:23.040964    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:28:23.174938 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:28:23.174901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:28:48.837856 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:28:48.837827    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:28:48.838398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:28:48.837861    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:29:17.011805 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:29:17.011768    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:30:40.454232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:30:40.454186    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:11.625356 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:11.625329    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:20.317354 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:20.317321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:27.715673 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:27.715639    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:32.842510 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:32.842474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:35.772466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:35.772422    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:35.898757 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:35.898708    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:36.230269 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:36.230230    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:42.601890 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:42.601815    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:42.704694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:42.704667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:43.049259 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:43.049228    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:43.236993 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:43.236913    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:44.098084 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.098047    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:44.197302 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.197264    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:44.336155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.336119    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"]
Jul 10 14:32:44.339310 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.339294    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.343587 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.343562    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-dc35352e-serving-cert\""
Jul 10 14:32:44.348387 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.348318    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.348387 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.348362    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcgq7\" (UniqueName: \"kubernetes.io/projected/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-kube-api-access-qcgq7\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.348580 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.348440    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-openshift-service-ca\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.369518 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.369492    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"]
Jul 10 14:32:44.430129 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.430092    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:44.449371 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.449338    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-openshift-service-ca\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.449517 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.449378    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.449517 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:32:44.449468    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-serving-cert: secret "maas-api-e2e-ait-dc35352e-serving-cert" not found
Jul 10 14:32:44.449626 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.449509    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgq7\" (UniqueName: \"kubernetes.io/projected/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-kube-api-access-qcgq7\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.449626 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:32:44.449522    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls podName:e96b7e99-92b2-492b-a396-d5d6a75c4f3f nodeName:}" failed. No retries permitted until 2026-07-10 14:32:44.949505811 +0000 UTC m=+1436.575231720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls") pod "maas-api-e2e-ait-dc35352e-5cb666587b-k7crj" (UID: "e96b7e99-92b2-492b-a396-d5d6a75c4f3f") : secret "maas-api-e2e-ait-dc35352e-serving-cert" not found
Jul 10 14:32:44.449990 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.449974    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-openshift-service-ca\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.472403 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.472367    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgq7\" (UniqueName: \"kubernetes.io/projected/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-kube-api-access-qcgq7\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.758078 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.757998    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:44.953042 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.953003    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:44.955357 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:44.955325    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls\") pod \"maas-api-e2e-ait-dc35352e-5cb666587b-k7crj\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:45.021185 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:45.021099    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:45.233695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:45.233664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:45.249692 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:45.249659    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:45.386677 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:45.386653    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"]
Jul 10 14:32:45.388538 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:32:45.388510    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96b7e99_92b2_492b_a396_d5d6a75c4f3f.slice/crio-58fb89091e5ff813ecb741cb45a86e8a65c9c98203d88e8717934060ca9ec114 WatchSource:0}: Error finding container 58fb89091e5ff813ecb741cb45a86e8a65c9c98203d88e8717934060ca9ec114: Status 404 returned error can't find the container with id 58fb89091e5ff813ecb741cb45a86e8a65c9c98203d88e8717934060ca9ec114
Jul 10 14:32:45.389835 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:45.389816    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:32:46.366704 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:46.366616    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj" event={"ID":"e96b7e99-92b2-492b-a396-d5d6a75c4f3f","Type":"ContainerStarted","Data":"1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5"}
Jul 10 14:32:46.366704 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:46.366678    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj" event={"ID":"e96b7e99-92b2-492b-a396-d5d6a75c4f3f","Type":"ContainerStarted","Data":"58fb89091e5ff813ecb741cb45a86e8a65c9c98203d88e8717934060ca9ec114"}
Jul 10 14:32:46.367131 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:46.366772    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:46.390366 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:46.390318    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj" podStartSLOduration=1.6883789230000001 podStartE2EDuration="2.390300438s" totalImagesPullingTime="701.921515ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:32:46.389059516 +0000 UTC m=+1438.014785448" watchObservedRunningTime="2026-07-10 14:32:46.390300438 +0000 UTC m=+1438.016026367"
Jul 10 14:32:47.205741 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:47.205693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:49.156572 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:49.156537    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:49.366104 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:49.366071    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:49.476070 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:49.475980    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:49.827964 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:49.827927    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"]
Jul 10 14:32:49.828176 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:49.828142    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj" podUID="e96b7e99-92b2-492b-a396-d5d6a75c4f3f" containerName="maas-api" containerID="cri-o://1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5" gracePeriod=30
Jul 10 14:32:49.900537 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:49.900509    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:50.070090 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.070064    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:50.090015 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.089933    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-openshift-service-ca\") pod \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") "
Jul 10 14:32:50.090015 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.089972    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcgq7\" (UniqueName: \"kubernetes.io/projected/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-kube-api-access-qcgq7\") pod \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") "
Jul 10 14:32:50.090015 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.090011    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls\") pod \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\" (UID: \"e96b7e99-92b2-492b-a396-d5d6a75c4f3f\") "
Jul 10 14:32:50.090331 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.090311    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e96b7e99-92b2-492b-a396-d5d6a75c4f3f" (UID: "e96b7e99-92b2-492b-a396-d5d6a75c4f3f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:32:50.092221 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.092194    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e96b7e99-92b2-492b-a396-d5d6a75c4f3f" (UID: "e96b7e99-92b2-492b-a396-d5d6a75c4f3f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:32:50.092316 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.092207    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-kube-api-access-qcgq7" (OuterVolumeSpecName: "kube-api-access-qcgq7") pod "e96b7e99-92b2-492b-a396-d5d6a75c4f3f" (UID: "e96b7e99-92b2-492b-a396-d5d6a75c4f3f"). InnerVolumeSpecName "kube-api-access-qcgq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:32:50.190727 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.190698    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:32:50.190727 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.190734    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:32:50.191141 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.190744    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcgq7\" (UniqueName: \"kubernetes.io/projected/e96b7e99-92b2-492b-a396-d5d6a75c4f3f-kube-api-access-qcgq7\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:32:50.384601 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.384504    2584 generic.go:356] "Generic (PLEG): container finished" podID="e96b7e99-92b2-492b-a396-d5d6a75c4f3f" containerID="1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5" exitCode=0
Jul 10 14:32:50.384601 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.384548    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"
Jul 10 14:32:50.384832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.384596    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj" event={"ID":"e96b7e99-92b2-492b-a396-d5d6a75c4f3f","Type":"ContainerDied","Data":"1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5"}
Jul 10 14:32:50.384832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.384648    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj" event={"ID":"e96b7e99-92b2-492b-a396-d5d6a75c4f3f","Type":"ContainerDied","Data":"58fb89091e5ff813ecb741cb45a86e8a65c9c98203d88e8717934060ca9ec114"}
Jul 10 14:32:50.384832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.384671    2584 scope.go:117] "RemoveContainer" containerID="1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5"
Jul 10 14:32:50.393247 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.393230    2584 scope.go:117] "RemoveContainer" containerID="1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5"
Jul 10 14:32:50.393500 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:32:50.393483    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5\": container with ID starting with 1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5 not found: ID does not exist" containerID="1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5"
Jul 10 14:32:50.393545 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.393507    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5"} err="failed to get container status \"1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5\": rpc error: code = NotFound desc = could not find container \"1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5\": container with ID starting with 1179e3e9a92653162c773b13f513c99d5a8a9a6f04b94783ea0801b5f24376e5 not found: ID does not exist"
Jul 10 14:32:50.409394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.409365    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"]
Jul 10 14:32:50.416383 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.412589    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc35352e-5cb666587b-k7crj"]
Jul 10 14:32:50.770749 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.770700    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:50.920326 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:50.920283    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e96b7e99-92b2-492b-a396-d5d6a75c4f3f" path="/var/lib/kubelet/pods/e96b7e99-92b2-492b-a396-d5d6a75c4f3f/volumes"
Jul 10 14:32:55.792937 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:55.792904    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:55.894821 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:55.894789    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:56.073907 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:56.073875    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:56.141862 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:56.141830    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:57.229171 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.229135    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:57.293045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.293016    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"]
Jul 10 14:32:57.293381 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.293370    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e96b7e99-92b2-492b-a396-d5d6a75c4f3f" containerName="maas-api"
Jul 10 14:32:57.293429 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.293383    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e96b7e99-92b2-492b-a396-d5d6a75c4f3f" containerName="maas-api"
Jul 10 14:32:57.293462 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.293443    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e96b7e99-92b2-492b-a396-d5d6a75c4f3f" containerName="maas-api"
Jul 10 14:32:57.298083 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.298063    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.301056 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.301035    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f7ca71ee-serving-cert\""
Jul 10 14:32:57.314253 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.314224    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"]
Jul 10 14:32:57.351701 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.351670    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27z8m\" (UniqueName: \"kubernetes.io/projected/783d08da-cd1b-4c6f-8c6e-f1917417c69a-kube-api-access-27z8m\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.351874 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.351709    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/783d08da-cd1b-4c6f-8c6e-f1917417c69a-openshift-service-ca\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.351874 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.351822    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.388534 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.388502    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:57.452662 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.452630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/783d08da-cd1b-4c6f-8c6e-f1917417c69a-openshift-service-ca\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.452871 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.452679    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.452871 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.452765    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27z8m\" (UniqueName: \"kubernetes.io/projected/783d08da-cd1b-4c6f-8c6e-f1917417c69a-kube-api-access-27z8m\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.452871 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:32:57.452847    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-serving-cert: secret "maas-api-e2e-ait-f7ca71ee-serving-cert" not found
Jul 10 14:32:57.453038 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:32:57.452932    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls podName:783d08da-cd1b-4c6f-8c6e-f1917417c69a nodeName:}" failed. No retries permitted until 2026-07-10 14:32:57.952916654 +0000 UTC m=+1449.578642562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls") pod "maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc" (UID: "783d08da-cd1b-4c6f-8c6e-f1917417c69a") : secret "maas-api-e2e-ait-f7ca71ee-serving-cert" not found
Jul 10 14:32:57.453362 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.453339    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/783d08da-cd1b-4c6f-8c6e-f1917417c69a-openshift-service-ca\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.469563 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.469536    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27z8m\" (UniqueName: \"kubernetes.io/projected/783d08da-cd1b-4c6f-8c6e-f1917417c69a-kube-api-access-27z8m\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.549013 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.548931    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:57.651381 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.651347    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:57.958441 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.958401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:57.960668 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:57.960638    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls\") pod \"maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:58.017828 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:58.017788    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:58.120647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:58.120614    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:58.211143 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:58.211056    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:58.351964 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:58.351884    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"]
Jul 10 14:32:58.354217 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:32:58.354193    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783d08da_cd1b_4c6f_8c6e_f1917417c69a.slice/crio-180fc72ee6a81521f8c4093dc320e73cc113bd444856adf052ecc1c3d3908b22 WatchSource:0}: Error finding container 180fc72ee6a81521f8c4093dc320e73cc113bd444856adf052ecc1c3d3908b22: Status 404 returned error can't find the container with id 180fc72ee6a81521f8c4093dc320e73cc113bd444856adf052ecc1c3d3908b22
Jul 10 14:32:58.422782 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:58.422744    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc" event={"ID":"783d08da-cd1b-4c6f-8c6e-f1917417c69a","Type":"ContainerStarted","Data":"180fc72ee6a81521f8c4093dc320e73cc113bd444856adf052ecc1c3d3908b22"}
Jul 10 14:32:59.228074 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:59.228043    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:32:59.428251 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:59.428220    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc" event={"ID":"783d08da-cd1b-4c6f-8c6e-f1917417c69a","Type":"ContainerStarted","Data":"eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669"}
Jul 10 14:32:59.428614 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:59.428349    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:32:59.456946 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:32:59.456855    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc" podStartSLOduration=2.063871111 podStartE2EDuration="2.45684167s" totalImagesPullingTime="392.970559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:32:59.456186618 +0000 UTC m=+1451.081912547" watchObservedRunningTime="2026-07-10 14:32:59.45684167 +0000 UTC m=+1451.082567599"
Jul 10 14:33:02.391948 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:02.391917    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:02.880565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:02.880532    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"]
Jul 10 14:33:02.880741 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:02.880702    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc" podUID="783d08da-cd1b-4c6f-8c6e-f1917417c69a" containerName="maas-api" containerID="cri-o://eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669" gracePeriod=30
Jul 10 14:33:02.952667 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:02.952633    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:03.123841 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.123815    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:33:03.204690 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.204619    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/783d08da-cd1b-4c6f-8c6e-f1917417c69a-openshift-service-ca\") pod \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") "
Jul 10 14:33:03.204690 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.204664    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27z8m\" (UniqueName: \"kubernetes.io/projected/783d08da-cd1b-4c6f-8c6e-f1917417c69a-kube-api-access-27z8m\") pod \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") "
Jul 10 14:33:03.204916 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.204700    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls\") pod \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\" (UID: \"783d08da-cd1b-4c6f-8c6e-f1917417c69a\") "
Jul 10 14:33:03.205049 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.205025    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/783d08da-cd1b-4c6f-8c6e-f1917417c69a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "783d08da-cd1b-4c6f-8c6e-f1917417c69a" (UID: "783d08da-cd1b-4c6f-8c6e-f1917417c69a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:33:03.206712 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.206683    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/783d08da-cd1b-4c6f-8c6e-f1917417c69a-kube-api-access-27z8m" (OuterVolumeSpecName: "kube-api-access-27z8m") pod "783d08da-cd1b-4c6f-8c6e-f1917417c69a" (UID: "783d08da-cd1b-4c6f-8c6e-f1917417c69a"). InnerVolumeSpecName "kube-api-access-27z8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:33:03.206712 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.206684    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "783d08da-cd1b-4c6f-8c6e-f1917417c69a" (UID: "783d08da-cd1b-4c6f-8c6e-f1917417c69a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:33:03.305466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.305437    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/783d08da-cd1b-4c6f-8c6e-f1917417c69a-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:33:03.305466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.305462    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27z8m\" (UniqueName: \"kubernetes.io/projected/783d08da-cd1b-4c6f-8c6e-f1917417c69a-kube-api-access-27z8m\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:33:03.305466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.305472    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/783d08da-cd1b-4c6f-8c6e-f1917417c69a-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:33:03.446897 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.446853    2584 generic.go:356] "Generic (PLEG): container finished" podID="783d08da-cd1b-4c6f-8c6e-f1917417c69a" containerID="eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669" exitCode=0
Jul 10 14:33:03.446897 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.446908    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"
Jul 10 14:33:03.447418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.446923    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc" event={"ID":"783d08da-cd1b-4c6f-8c6e-f1917417c69a","Type":"ContainerDied","Data":"eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669"}
Jul 10 14:33:03.447418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.446978    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc" event={"ID":"783d08da-cd1b-4c6f-8c6e-f1917417c69a","Type":"ContainerDied","Data":"180fc72ee6a81521f8c4093dc320e73cc113bd444856adf052ecc1c3d3908b22"}
Jul 10 14:33:03.447418 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.447007    2584 scope.go:117] "RemoveContainer" containerID="eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669"
Jul 10 14:33:03.456032 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.456012    2584 scope.go:117] "RemoveContainer" containerID="eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669"
Jul 10 14:33:03.456320 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:33:03.456297    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669\": container with ID starting with eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669 not found: ID does not exist" containerID="eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669"
Jul 10 14:33:03.456409 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.456330    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669"} err="failed to get container status \"eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669\": rpc error: code = NotFound desc = could not find container \"eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669\": container with ID starting with eea0b7e0cc1ba73448b85e63b56da6351447014c33e56f4d709647d0a03f0669 not found: ID does not exist"
Jul 10 14:33:03.474551 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.474517    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"]
Jul 10 14:33:03.477013 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:03.476993    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7ca71ee-6bd746bbff-zdtzc"]
Jul 10 14:33:04.921330 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:04.921297    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="783d08da-cd1b-4c6f-8c6e-f1917417c69a" path="/var/lib/kubelet/pods/783d08da-cd1b-4c6f-8c6e-f1917417c69a/volumes"
Jul 10 14:33:08.565933 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:08.565906    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:08.671008 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:08.670976    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:14.803657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:14.803601    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:14.909970 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:14.909932    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:15.190961 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:15.190927    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:15.971124 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:15.971086    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:16.072128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.072083    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:16.213445 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.213358    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:16.309657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.309622    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"]
Jul 10 14:33:16.309982 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.309970    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="783d08da-cd1b-4c6f-8c6e-f1917417c69a" containerName="maas-api"
Jul 10 14:33:16.310052 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.309983    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="783d08da-cd1b-4c6f-8c6e-f1917417c69a" containerName="maas-api"
Jul 10 14:33:16.310052 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.310044    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="783d08da-cd1b-4c6f-8c6e-f1917417c69a" containerName="maas-api"
Jul 10 14:33:16.314125 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.314107    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.319322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.319296    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c6adcb10-serving-cert\""
Jul 10 14:33:16.339771 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.339741    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"]
Jul 10 14:33:16.382364 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.382313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:16.421923 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.421890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16cf006b-2558-4c06-9ac6-8a61bf317b4e-openshift-service-ca\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.422094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.421947    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px5tf\" (UniqueName: \"kubernetes.io/projected/16cf006b-2558-4c06-9ac6-8a61bf317b4e-kube-api-access-px5tf\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.422094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.422012    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16cf006b-2558-4c06-9ac6-8a61bf317b4e-maas-api-tls\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.520936 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.520850    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:16.523081 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.523062    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px5tf\" (UniqueName: \"kubernetes.io/projected/16cf006b-2558-4c06-9ac6-8a61bf317b4e-kube-api-access-px5tf\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.523146 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.523093    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16cf006b-2558-4c06-9ac6-8a61bf317b4e-maas-api-tls\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.523181 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.523151    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16cf006b-2558-4c06-9ac6-8a61bf317b4e-openshift-service-ca\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.523758 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.523740    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16cf006b-2558-4c06-9ac6-8a61bf317b4e-openshift-service-ca\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.525447 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.525430    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16cf006b-2558-4c06-9ac6-8a61bf317b4e-maas-api-tls\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.540163 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.540129    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px5tf\" (UniqueName: \"kubernetes.io/projected/16cf006b-2558-4c06-9ac6-8a61bf317b4e-kube-api-access-px5tf\") pod \"maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.624303 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.624262    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:16.780841 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:16.780805    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"]
Jul 10 14:33:16.782388 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:33:16.782361    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16cf006b_2558_4c06_9ac6_8a61bf317b4e.slice/crio-2b5b815bd82c45b2b19bf069c9ad71fa7fa60e8a0b0ae340970766fee17bf15c WatchSource:0}: Error finding container 2b5b815bd82c45b2b19bf069c9ad71fa7fa60e8a0b0ae340970766fee17bf15c: Status 404 returned error can't find the container with id 2b5b815bd82c45b2b19bf069c9ad71fa7fa60e8a0b0ae340970766fee17bf15c
Jul 10 14:33:17.137328 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:17.137288    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:17.248139 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:17.248114    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:17.510565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:17.510476    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn" event={"ID":"16cf006b-2558-4c06-9ac6-8a61bf317b4e","Type":"ContainerStarted","Data":"dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e"}
Jul 10 14:33:17.510565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:17.510525    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn" event={"ID":"16cf006b-2558-4c06-9ac6-8a61bf317b4e","Type":"ContainerStarted","Data":"2b5b815bd82c45b2b19bf069c9ad71fa7fa60e8a0b0ae340970766fee17bf15c"}
Jul 10 14:33:17.510797 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:17.510623    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:17.538330 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:17.538271    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn" podStartSLOduration=1.086254663 podStartE2EDuration="1.538251328s" totalImagesPullingTime="451.996665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:33:17.535425345 +0000 UTC m=+1469.161151275" watchObservedRunningTime="2026-07-10 14:33:17.538251328 +0000 UTC m=+1469.163977259"
Jul 10 14:33:20.009768 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:20.009707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:20.612074 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:20.612031    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:20.822947 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:20.822902    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:21.047079 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.046996    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:21.104504 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.104471    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"]
Jul 10 14:33:21.104675 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.104652    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn" podUID="16cf006b-2558-4c06-9ac6-8a61bf317b4e" containerName="maas-api" containerID="cri-o://dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e" gracePeriod=30
Jul 10 14:33:21.152369 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.152336    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:21.351317 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.351291    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:21.469544 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.469510    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16cf006b-2558-4c06-9ac6-8a61bf317b4e-openshift-service-ca\") pod \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") "
Jul 10 14:33:21.469702 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.469634    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16cf006b-2558-4c06-9ac6-8a61bf317b4e-maas-api-tls\") pod \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") "
Jul 10 14:33:21.469702 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.469656    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px5tf\" (UniqueName: \"kubernetes.io/projected/16cf006b-2558-4c06-9ac6-8a61bf317b4e-kube-api-access-px5tf\") pod \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\" (UID: \"16cf006b-2558-4c06-9ac6-8a61bf317b4e\") "
Jul 10 14:33:21.469944 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.469916    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16cf006b-2558-4c06-9ac6-8a61bf317b4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16cf006b-2558-4c06-9ac6-8a61bf317b4e" (UID: "16cf006b-2558-4c06-9ac6-8a61bf317b4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:33:21.471816 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.471786    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16cf006b-2558-4c06-9ac6-8a61bf317b4e-kube-api-access-px5tf" (OuterVolumeSpecName: "kube-api-access-px5tf") pod "16cf006b-2558-4c06-9ac6-8a61bf317b4e" (UID: "16cf006b-2558-4c06-9ac6-8a61bf317b4e"). InnerVolumeSpecName "kube-api-access-px5tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:33:21.471946 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.471883    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16cf006b-2558-4c06-9ac6-8a61bf317b4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16cf006b-2558-4c06-9ac6-8a61bf317b4e" (UID: "16cf006b-2558-4c06-9ac6-8a61bf317b4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:33:21.531789 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.531754    2584 generic.go:356] "Generic (PLEG): container finished" podID="16cf006b-2558-4c06-9ac6-8a61bf317b4e" containerID="dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e" exitCode=0
Jul 10 14:33:21.531949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.531801    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"
Jul 10 14:33:21.531949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.531837    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn" event={"ID":"16cf006b-2558-4c06-9ac6-8a61bf317b4e","Type":"ContainerDied","Data":"dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e"}
Jul 10 14:33:21.531949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.531888    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn" event={"ID":"16cf006b-2558-4c06-9ac6-8a61bf317b4e","Type":"ContainerDied","Data":"2b5b815bd82c45b2b19bf069c9ad71fa7fa60e8a0b0ae340970766fee17bf15c"}
Jul 10 14:33:21.531949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.531925    2584 scope.go:117] "RemoveContainer" containerID="dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e"
Jul 10 14:33:21.540968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.540950    2584 scope.go:117] "RemoveContainer" containerID="dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e"
Jul 10 14:33:21.541262 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:33:21.541240    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e\": container with ID starting with dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e not found: ID does not exist" containerID="dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e"
Jul 10 14:33:21.541361 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.541270    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e"} err="failed to get container status \"dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e\": rpc error: code = NotFound desc = could not find container \"dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e\": container with ID starting with dd40c5f134594928aa5c0c1814f6ef4c7233e0d741318e444d5386e2f874965e not found: ID does not exist"
Jul 10 14:33:21.556179 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.556149    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"]
Jul 10 14:33:21.560200 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.560177    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c6adcb10-548895cbff-zzmkn"]
Jul 10 14:33:21.570949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.570927    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16cf006b-2558-4c06-9ac6-8a61bf317b4e-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:33:21.571036 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.570952    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px5tf\" (UniqueName: \"kubernetes.io/projected/16cf006b-2558-4c06-9ac6-8a61bf317b4e-kube-api-access-px5tf\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:33:21.571036 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:21.570961    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16cf006b-2558-4c06-9ac6-8a61bf317b4e-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:33:22.698168 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:22.698137    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:22.920622 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:22.920591    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16cf006b-2558-4c06-9ac6-8a61bf317b4e" path="/var/lib/kubelet/pods/16cf006b-2558-4c06-9ac6-8a61bf317b4e/volumes"
Jul 10 14:33:27.416935 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:27.416902    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:27.531744 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:27.531688    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:27.866408 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:27.866371    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:27.968849 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:27.968812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:28.071963 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:28.071930    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:29.507596 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:29.507562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:35.663950 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:35.663919    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:35.772731 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:35.772678    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:35.868997 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:35.868963    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:35.983212 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:35.983124    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:36.445591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:36.445559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:33:48.864957 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:48.864930    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:33:48.865886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:33:48.865859    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:34:02.435370 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:02.435333    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:08.538685 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:08.538643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:08.660434 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:08.660399    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:08.809008 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:08.808905    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:24.682443 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:24.682363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:30.897070 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:30.897031    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:30.964805 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:30.964777    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:31.057634 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:31.057603    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:31.164558 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:31.164469    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:34.762438 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:34.762401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:41.640618 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:41.640579    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:42.443025 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:42.442988    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:48.472998 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:48.472961    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:48.577595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:48.577566    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:48.676987 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:48.676952    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:48.770606 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:48.770530    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:49.571092 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:49.571046    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:49.659035 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:49.658997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:49.761157 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:49.761124    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:51.451048 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:51.451007    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:57.629603 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:57.629571    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:57.749672 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:57.749639    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:34:58.291389 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:34:58.291351    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:05.857015 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:05.856974    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:05.957154 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:05.957125    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:06.101686 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:06.101654    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:06.455638 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:06.455599    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:06.682760 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:06.682705    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:09.735385 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.735355    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:09.826777 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.826750    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:09.885454 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.885426    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"]
Jul 10 14:35:09.885786 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.885773    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16cf006b-2558-4c06-9ac6-8a61bf317b4e" containerName="maas-api"
Jul 10 14:35:09.885842 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.885787    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="16cf006b-2558-4c06-9ac6-8a61bf317b4e" containerName="maas-api"
Jul 10 14:35:09.885842 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.885840    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="16cf006b-2558-4c06-9ac6-8a61bf317b4e" containerName="maas-api"
Jul 10 14:35:09.888674 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.888658    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:09.896091 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.896072    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8a12160b-serving-cert\""
Jul 10 14:35:09.907123 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.907099    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"]
Jul 10 14:35:09.982287 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:09.982256    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:10.030406 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.030336    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51046a33-ba81-47de-8a32-1f11d1192331-openshift-service-ca\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.030406 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.030401    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6bwn\" (UniqueName: \"kubernetes.io/projected/51046a33-ba81-47de-8a32-1f11d1192331-kube-api-access-b6bwn\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.030566 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.030439    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51046a33-ba81-47de-8a32-1f11d1192331-maas-api-tls\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.087730 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.087693    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:10.130820 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.130795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6bwn\" (UniqueName: \"kubernetes.io/projected/51046a33-ba81-47de-8a32-1f11d1192331-kube-api-access-b6bwn\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.130952 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.130844    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51046a33-ba81-47de-8a32-1f11d1192331-maas-api-tls\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.130952 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.130945    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51046a33-ba81-47de-8a32-1f11d1192331-openshift-service-ca\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.131505 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.131487    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51046a33-ba81-47de-8a32-1f11d1192331-openshift-service-ca\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.133194 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.133173    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51046a33-ba81-47de-8a32-1f11d1192331-maas-api-tls\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.143300 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.143282    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6bwn\" (UniqueName: \"kubernetes.io/projected/51046a33-ba81-47de-8a32-1f11d1192331-kube-api-access-b6bwn\") pod \"maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.198711 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.198685    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:10.326574 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.326551    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"]
Jul 10 14:35:10.328634 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:35:10.328606    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51046a33_ba81_47de_8a32_1f11d1192331.slice/crio-85396b3d3bb610d091f4ba11b4eee2489575f9074380be60274c9524b875d4f6 WatchSource:0}: Error finding container 85396b3d3bb610d091f4ba11b4eee2489575f9074380be60274c9524b875d4f6: Status 404 returned error can't find the container with id 85396b3d3bb610d091f4ba11b4eee2489575f9074380be60274c9524b875d4f6
Jul 10 14:35:10.491701 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.491667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:10.589834 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:10.589745    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:11.024335 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:11.024247    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z" event={"ID":"51046a33-ba81-47de-8a32-1f11d1192331","Type":"ContainerStarted","Data":"85396b3d3bb610d091f4ba11b4eee2489575f9074380be60274c9524b875d4f6"}
Jul 10 14:35:12.030354 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:12.030320    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z" event={"ID":"51046a33-ba81-47de-8a32-1f11d1192331","Type":"ContainerStarted","Data":"4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87"}
Jul 10 14:35:12.030817 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:12.030411    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:12.106463 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:12.106415    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z" podStartSLOduration=2.2670331360000002 podStartE2EDuration="3.106401305s" totalImagesPullingTime="839.368169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:12.104737396 +0000 UTC m=+1583.730463328" watchObservedRunningTime="2026-07-10 14:35:12.106401305 +0000 UTC m=+1583.732127236"
Jul 10 14:35:14.205232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:14.205196    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:14.896267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:14.896231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:14.995016 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:14.994992    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:15.332107 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:15.332076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:18.041053 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:18.041026    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:21.778853 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:21.778821    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:21.876183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:21.876151    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:22.188686 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.188640    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:22.549508 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.549426    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:22.561830 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.561804    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"]
Jul 10 14:35:22.562002 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.561981    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z" podUID="51046a33-ba81-47de-8a32-1f11d1192331" containerName="maas-api" containerID="cri-o://4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87" gracePeriod=30
Jul 10 14:35:22.647521 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.647483    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:22.808907 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.808856    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:22.836486 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.836458    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51046a33-ba81-47de-8a32-1f11d1192331-openshift-service-ca\") pod \"51046a33-ba81-47de-8a32-1f11d1192331\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") "
Jul 10 14:35:22.836604 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.836537    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6bwn\" (UniqueName: \"kubernetes.io/projected/51046a33-ba81-47de-8a32-1f11d1192331-kube-api-access-b6bwn\") pod \"51046a33-ba81-47de-8a32-1f11d1192331\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") "
Jul 10 14:35:22.836604 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.836576    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51046a33-ba81-47de-8a32-1f11d1192331-maas-api-tls\") pod \"51046a33-ba81-47de-8a32-1f11d1192331\" (UID: \"51046a33-ba81-47de-8a32-1f11d1192331\") "
Jul 10 14:35:22.836857 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.836833    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51046a33-ba81-47de-8a32-1f11d1192331-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "51046a33-ba81-47de-8a32-1f11d1192331" (UID: "51046a33-ba81-47de-8a32-1f11d1192331"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:35:22.838543 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.838514    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51046a33-ba81-47de-8a32-1f11d1192331-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "51046a33-ba81-47de-8a32-1f11d1192331" (UID: "51046a33-ba81-47de-8a32-1f11d1192331"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:35:22.838962 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.838934    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51046a33-ba81-47de-8a32-1f11d1192331-kube-api-access-b6bwn" (OuterVolumeSpecName: "kube-api-access-b6bwn") pod "51046a33-ba81-47de-8a32-1f11d1192331" (UID: "51046a33-ba81-47de-8a32-1f11d1192331"). InnerVolumeSpecName "kube-api-access-b6bwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:35:22.938087 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.938064    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51046a33-ba81-47de-8a32-1f11d1192331-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:35:22.938087 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.938086    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6bwn\" (UniqueName: \"kubernetes.io/projected/51046a33-ba81-47de-8a32-1f11d1192331-kube-api-access-b6bwn\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:35:22.938232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:22.938096    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51046a33-ba81-47de-8a32-1f11d1192331-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:35:23.080284 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.080249    2584 generic.go:356] "Generic (PLEG): container finished" podID="51046a33-ba81-47de-8a32-1f11d1192331" containerID="4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87" exitCode=0
Jul 10 14:35:23.080460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.080307    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"
Jul 10 14:35:23.080460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.080331    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z" event={"ID":"51046a33-ba81-47de-8a32-1f11d1192331","Type":"ContainerDied","Data":"4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87"}
Jul 10 14:35:23.080460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.080399    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z" event={"ID":"51046a33-ba81-47de-8a32-1f11d1192331","Type":"ContainerDied","Data":"85396b3d3bb610d091f4ba11b4eee2489575f9074380be60274c9524b875d4f6"}
Jul 10 14:35:23.080460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.080419    2584 scope.go:117] "RemoveContainer" containerID="4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87"
Jul 10 14:35:23.088667 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.088646    2584 scope.go:117] "RemoveContainer" containerID="4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87"
Jul 10 14:35:23.088915 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:35:23.088899    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87\": container with ID starting with 4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87 not found: ID does not exist" containerID="4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87"
Jul 10 14:35:23.088991 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.088922    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87"} err="failed to get container status \"4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87\": rpc error: code = NotFound desc = could not find container \"4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87\": container with ID starting with 4b3c88bad1b802040db5943baaa1423a4df322263fc31a8eb50fdc06f0603d87 not found: ID does not exist"
Jul 10 14:35:23.098370 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.098350    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"]
Jul 10 14:35:23.103145 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:23.103126    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a12160b-6d57b88458-nkr7z"]
Jul 10 14:35:24.920539 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:24.920506    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51046a33-ba81-47de-8a32-1f11d1192331" path="/var/lib/kubelet/pods/51046a33-ba81-47de-8a32-1f11d1192331/volumes"
Jul 10 14:35:28.675984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:28.675958    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:28.760675 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:28.760610    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:28.949668 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:28.949546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:29.187003 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:29.186968    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:29.327285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:29.327243    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:29.452676 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:29.452647    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:29.587352 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:29.587268    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:35.543204 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:35.543165    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:35.649514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:35.649478    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:35.914548 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:35.914516    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:36.055664 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:36.055609    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:42.814211 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:42.814177    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:42.929455 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:42.929423    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:49.792294 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:49.792248    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:50.322411 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:50.322373    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:50.508218 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:50.508188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.034702 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.034670    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.193232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.193203    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"]
Jul 10 14:35:54.193556 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.193541    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51046a33-ba81-47de-8a32-1f11d1192331" containerName="maas-api"
Jul 10 14:35:54.193598 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.193559    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="51046a33-ba81-47de-8a32-1f11d1192331" containerName="maas-api"
Jul 10 14:35:54.193649 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.193639    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="51046a33-ba81-47de-8a32-1f11d1192331" containerName="maas-api"
Jul 10 14:35:54.196432 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.196416    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.199607 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.199590    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-59512b8b-serving-cert\""
Jul 10 14:35:54.251304 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.251266    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.252261 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.252238    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"]
Jul 10 14:35:54.315623 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.315598    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c374f93-c084-4cff-9244-22ea8239ee34-openshift-service-ca\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.315810 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.315646    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.315810 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.315743    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzmqw\" (UniqueName: \"kubernetes.io/projected/1c374f93-c084-4cff-9244-22ea8239ee34-kube-api-access-lzmqw\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.416954 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.416921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.417120 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.416970    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzmqw\" (UniqueName: \"kubernetes.io/projected/1c374f93-c084-4cff-9244-22ea8239ee34-kube-api-access-lzmqw\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.417120 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.417031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c374f93-c084-4cff-9244-22ea8239ee34-openshift-service-ca\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.417120 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:35:54.417072    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-serving-cert: secret "maas-api-e2e-mt-59512b8b-serving-cert" not found
Jul 10 14:35:54.417251 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:35:54.417154    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls podName:1c374f93-c084-4cff-9244-22ea8239ee34 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:54.917135638 +0000 UTC m=+1626.542861548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls") pod "maas-api-e2e-mt-59512b8b-78b4d79947-tnq86" (UID: "1c374f93-c084-4cff-9244-22ea8239ee34") : secret "maas-api-e2e-mt-59512b8b-serving-cert" not found
Jul 10 14:35:54.417681 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.417663    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c374f93-c084-4cff-9244-22ea8239ee34-openshift-service-ca\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.419053 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.419029    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.433388 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.433361    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzmqw\" (UniqueName: \"kubernetes.io/projected/1c374f93-c084-4cff-9244-22ea8239ee34-kube-api-access-lzmqw\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.547441 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.547408    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:54.921510 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.921479    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:54.923840 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:54.923817    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls\") pod \"maas-api-e2e-mt-59512b8b-78b4d79947-tnq86\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:55.014018 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:55.013984    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:55.105706 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:55.105672    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:55.113448 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:55.113421    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:55.237874 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:55.237850    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"]
Jul 10 14:35:55.240476 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:35:55.240449    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c374f93_c084_4cff_9244_22ea8239ee34.slice/crio-10212bdd295e04178c48d1579ff70ed6bf921178b800dbfa54a15463712ae421 WatchSource:0}: Error finding container 10212bdd295e04178c48d1579ff70ed6bf921178b800dbfa54a15463712ae421: Status 404 returned error can't find the container with id 10212bdd295e04178c48d1579ff70ed6bf921178b800dbfa54a15463712ae421
Jul 10 14:35:55.945183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:55.945152    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:56.049196 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:56.049121    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:56.231917 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:56.231881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86" event={"ID":"1c374f93-c084-4cff-9244-22ea8239ee34","Type":"ContainerStarted","Data":"c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d"}
Jul 10 14:35:56.232362 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:56.231935    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86" event={"ID":"1c374f93-c084-4cff-9244-22ea8239ee34","Type":"ContainerStarted","Data":"10212bdd295e04178c48d1579ff70ed6bf921178b800dbfa54a15463712ae421"}
Jul 10 14:35:56.232362 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:56.231976    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:35:56.260107 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:56.260064    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86" podStartSLOduration=1.790570925 podStartE2EDuration="2.26004938s" totalImagesPullingTime="469.478455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:56.259156513 +0000 UTC m=+1627.884882443" watchObservedRunningTime="2026-07-10 14:35:56.26004938 +0000 UTC m=+1627.885775311"
Jul 10 14:35:56.932137 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:56.932099    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:57.027325 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:57.027295    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:35:58.434137 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:35:58.434108    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:02.242263 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:02.242224    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:36:09.569708 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.569672    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"]
Jul 10 14:36:09.572966 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.572935    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.576240 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.576216    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-59512b8b\"/\"openshift-service-ca.crt\""
Jul 10 14:36:09.577996 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.577974    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-59512b8b\"/\"kube-root-ca.crt\""
Jul 10 14:36:09.578111 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.578006    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-59512b8b\"/\"e2e-lifecycle-model-59512b8b-kserve-self-signed-certs\""
Jul 10 14:36:09.578111 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.578041    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-59512b8b\"/\"default-dockercfg-xgjcr\""
Jul 10 14:36:09.586151 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.586127    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"]
Jul 10 14:36:09.609591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.609559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:09.652073 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.652039    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-tmp-dir\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.652241 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.652079    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1e0f4e-4537-4fda-9321-b930d128649c-tls-certs\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.652241 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.652114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-dshm\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.652241 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.652160    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5dm2\" (UniqueName: \"kubernetes.io/projected/2c1e0f4e-4537-4fda-9321-b930d128649c-kube-api-access-t5dm2\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.652241 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.652188    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-model-cache\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.652241 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.652212    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-home\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753024 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.752984    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-tmp-dir\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753024 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753027    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1e0f4e-4537-4fda-9321-b930d128649c-tls-certs\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753152    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-dshm\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753198    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5dm2\" (UniqueName: \"kubernetes.io/projected/2c1e0f4e-4537-4fda-9321-b930d128649c-kube-api-access-t5dm2\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753305 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753241    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-model-cache\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753305 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753267    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-home\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753419 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753395    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-tmp-dir\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753619 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-model-cache\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.753692 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.753632    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-home\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.755354 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.755340    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-dshm\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.755608 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.755591    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1e0f4e-4537-4fda-9321-b930d128649c-tls-certs\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.765426 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.765404    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5dm2\" (UniqueName: \"kubernetes.io/projected/2c1e0f4e-4537-4fda-9321-b930d128649c-kube-api-access-t5dm2\") pod \"e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") " pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:09.882744 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:09.882627    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:10.018490 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:10.018463    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"]
Jul 10 14:36:10.020139 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:36:10.020118    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c1e0f4e_4537_4fda_9321_b930d128649c.slice/crio-9d6b55b416f42ad32c12953fed288f78722df2b943124bb594d79934648fadd8 WatchSource:0}: Error finding container 9d6b55b416f42ad32c12953fed288f78722df2b943124bb594d79934648fadd8: Status 404 returned error can't find the container with id 9d6b55b416f42ad32c12953fed288f78722df2b943124bb594d79934648fadd8
Jul 10 14:36:10.294956 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:10.294861    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf" event={"ID":"2c1e0f4e-4537-4fda-9321-b930d128649c","Type":"ContainerStarted","Data":"21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669"}
Jul 10 14:36:10.294956 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:10.294917    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf" event={"ID":"2c1e0f4e-4537-4fda-9321-b930d128649c","Type":"ContainerStarted","Data":"9d6b55b416f42ad32c12953fed288f78722df2b943124bb594d79934648fadd8"}
Jul 10 14:36:10.315362 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:10.315308    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf" podStartSLOduration=1.315293141 podStartE2EDuration="1.315293141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:10.31366102 +0000 UTC m=+1641.939386953" watchObservedRunningTime="2026-07-10 14:36:10.315293141 +0000 UTC m=+1641.941019103"
Jul 10 14:36:19.882837 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:19.882806    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:19.883234 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:19.883084    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:19.895357 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:19.895336    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:20.351045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:20.351008    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:31.121631 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:31.121598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:31.686429 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:31.686393    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"]
Jul 10 14:36:31.686656 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:31.686630    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86" podUID="1c374f93-c084-4cff-9244-22ea8239ee34" containerName="maas-api" containerID="cri-o://c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d" gracePeriod=30
Jul 10 14:36:31.727787 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:31.727755    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:31.931829 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:31.931805    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:36:32.060368 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.060276    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c374f93-c084-4cff-9244-22ea8239ee34-openshift-service-ca\") pod \"1c374f93-c084-4cff-9244-22ea8239ee34\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") "
Jul 10 14:36:32.060368 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.060353    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls\") pod \"1c374f93-c084-4cff-9244-22ea8239ee34\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") "
Jul 10 14:36:32.060558 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.060386    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzmqw\" (UniqueName: \"kubernetes.io/projected/1c374f93-c084-4cff-9244-22ea8239ee34-kube-api-access-lzmqw\") pod \"1c374f93-c084-4cff-9244-22ea8239ee34\" (UID: \"1c374f93-c084-4cff-9244-22ea8239ee34\") "
Jul 10 14:36:32.060633 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.060610    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c374f93-c084-4cff-9244-22ea8239ee34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c374f93-c084-4cff-9244-22ea8239ee34" (UID: "1c374f93-c084-4cff-9244-22ea8239ee34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:36:32.062487 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.062456    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c374f93-c084-4cff-9244-22ea8239ee34-kube-api-access-lzmqw" (OuterVolumeSpecName: "kube-api-access-lzmqw") pod "1c374f93-c084-4cff-9244-22ea8239ee34" (UID: "1c374f93-c084-4cff-9244-22ea8239ee34"). InnerVolumeSpecName "kube-api-access-lzmqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:36:32.062487 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.062464    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c374f93-c084-4cff-9244-22ea8239ee34" (UID: "1c374f93-c084-4cff-9244-22ea8239ee34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:36:32.161388 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.161345    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c374f93-c084-4cff-9244-22ea8239ee34-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:32.161388 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.161380    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzmqw\" (UniqueName: \"kubernetes.io/projected/1c374f93-c084-4cff-9244-22ea8239ee34-kube-api-access-lzmqw\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:32.161388 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.161391    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c374f93-c084-4cff-9244-22ea8239ee34-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:32.406087 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.406051    2584 generic.go:356] "Generic (PLEG): container finished" podID="1c374f93-c084-4cff-9244-22ea8239ee34" containerID="c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d" exitCode=0
Jul 10 14:36:32.406295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.406092    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"
Jul 10 14:36:32.406295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.406136    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86" event={"ID":"1c374f93-c084-4cff-9244-22ea8239ee34","Type":"ContainerDied","Data":"c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d"}
Jul 10 14:36:32.406295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.406189    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86" event={"ID":"1c374f93-c084-4cff-9244-22ea8239ee34","Type":"ContainerDied","Data":"10212bdd295e04178c48d1579ff70ed6bf921178b800dbfa54a15463712ae421"}
Jul 10 14:36:32.406295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.406212    2584 scope.go:117] "RemoveContainer" containerID="c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d"
Jul 10 14:36:32.417453 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.417431    2584 scope.go:117] "RemoveContainer" containerID="c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d"
Jul 10 14:36:32.417937 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:36:32.417907    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d\": container with ID starting with c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d not found: ID does not exist" containerID="c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d"
Jul 10 14:36:32.418039 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.417947    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d"} err="failed to get container status \"c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d\": rpc error: code = NotFound desc = could not find container \"c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d\": container with ID starting with c55e251195ae63e2f498685732f62f627349682f2501b316ff625ca258bf4f1d not found: ID does not exist"
Jul 10 14:36:32.435870 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.435834    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"]
Jul 10 14:36:32.449052 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.449021    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-59512b8b-78b4d79947-tnq86"]
Jul 10 14:36:32.921155 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:32.921125    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c374f93-c084-4cff-9244-22ea8239ee34" path="/var/lib/kubelet/pods/1c374f93-c084-4cff-9244-22ea8239ee34/volumes"
Jul 10 14:36:33.808190 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:33.808154    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:37.171897 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:37.171866    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:42.175314 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:42.175274    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"]
Jul 10 14:36:42.365242 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:42.365210    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:36:57.195673 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.195612    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf" podUID="2c1e0f4e-4537-4fda-9321-b930d128649c" containerName="main" containerID="cri-o://21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669" gracePeriod=45
Jul 10 14:36:57.446007 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.445952    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:57.476200 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476174    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-model-cache\") pod \"2c1e0f4e-4537-4fda-9321-b930d128649c\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") "
Jul 10 14:36:57.476339 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476216    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5dm2\" (UniqueName: \"kubernetes.io/projected/2c1e0f4e-4537-4fda-9321-b930d128649c-kube-api-access-t5dm2\") pod \"2c1e0f4e-4537-4fda-9321-b930d128649c\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") "
Jul 10 14:36:57.476339 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476247    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1e0f4e-4537-4fda-9321-b930d128649c-tls-certs\") pod \"2c1e0f4e-4537-4fda-9321-b930d128649c\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") "
Jul 10 14:36:57.476339 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476288    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-tmp-dir\") pod \"2c1e0f4e-4537-4fda-9321-b930d128649c\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") "
Jul 10 14:36:57.476536 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476393    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-dshm\") pod \"2c1e0f4e-4537-4fda-9321-b930d128649c\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") "
Jul 10 14:36:57.476536 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476424    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-home\") pod \"2c1e0f4e-4537-4fda-9321-b930d128649c\" (UID: \"2c1e0f4e-4537-4fda-9321-b930d128649c\") "
Jul 10 14:36:57.476670 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476548    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c1e0f4e-4537-4fda-9321-b930d128649c" (UID: "2c1e0f4e-4537-4fda-9321-b930d128649c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:36:57.476760 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476745    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-home" (OuterVolumeSpecName: "home") pod "2c1e0f4e-4537-4fda-9321-b930d128649c" (UID: "2c1e0f4e-4537-4fda-9321-b930d128649c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:36:57.476825 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476786    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-model-cache\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:57.476963 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.476932    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c1e0f4e-4537-4fda-9321-b930d128649c" (UID: "2c1e0f4e-4537-4fda-9321-b930d128649c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:36:57.478899 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.478861    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c1e0f4e-4537-4fda-9321-b930d128649c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c1e0f4e-4537-4fda-9321-b930d128649c" (UID: "2c1e0f4e-4537-4fda-9321-b930d128649c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:36:57.479101 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.479081    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-dshm" (OuterVolumeSpecName: "dshm") pod "2c1e0f4e-4537-4fda-9321-b930d128649c" (UID: "2c1e0f4e-4537-4fda-9321-b930d128649c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:36:57.479154 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.479110    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c1e0f4e-4537-4fda-9321-b930d128649c-kube-api-access-t5dm2" (OuterVolumeSpecName: "kube-api-access-t5dm2") pod "2c1e0f4e-4537-4fda-9321-b930d128649c" (UID: "2c1e0f4e-4537-4fda-9321-b930d128649c"). InnerVolumeSpecName "kube-api-access-t5dm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:36:57.526232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.526201    2584 generic.go:356] "Generic (PLEG): container finished" podID="2c1e0f4e-4537-4fda-9321-b930d128649c" containerID="21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669" exitCode=0
Jul 10 14:36:57.526354 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.526239    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"
Jul 10 14:36:57.526354 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.526289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf" event={"ID":"2c1e0f4e-4537-4fda-9321-b930d128649c","Type":"ContainerDied","Data":"21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669"}
Jul 10 14:36:57.526354 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.526335    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf" event={"ID":"2c1e0f4e-4537-4fda-9321-b930d128649c","Type":"ContainerDied","Data":"9d6b55b416f42ad32c12953fed288f78722df2b943124bb594d79934648fadd8"}
Jul 10 14:36:57.526453 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.526356    2584 scope.go:117] "RemoveContainer" containerID="21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669"
Jul 10 14:36:57.534698 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.534682    2584 scope.go:117] "RemoveContainer" containerID="21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669"
Jul 10 14:36:57.534976 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:36:57.534959    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669\": container with ID starting with 21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669 not found: ID does not exist" containerID="21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669"
Jul 10 14:36:57.535042 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.534991    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669"} err="failed to get container status \"21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669\": rpc error: code = NotFound desc = could not find container \"21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669\": container with ID starting with 21fb81b58d049ea64936cd3aa134935f86e278559b3bec179e3bda2f48cd7669 not found: ID does not exist"
Jul 10 14:36:57.548863 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.548843    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"]
Jul 10 14:36:57.555474 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.555457    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-59512b8b/e2e-lifecycle-model-59512b8b-kserve-7f45b66ddc-8l5tf"]
Jul 10 14:36:57.577604 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.577584    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5dm2\" (UniqueName: \"kubernetes.io/projected/2c1e0f4e-4537-4fda-9321-b930d128649c-kube-api-access-t5dm2\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:57.577604 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.577603    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c1e0f4e-4537-4fda-9321-b930d128649c-tls-certs\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:57.577696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.577632    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-tmp-dir\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:57.577696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.577640    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-dshm\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:57.577696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:57.577649    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c1e0f4e-4537-4fda-9321-b930d128649c-home\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:36:58.920128 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:36:58.920096    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c1e0f4e-4537-4fda-9321-b930d128649c" path="/var/lib/kubelet/pods/2c1e0f4e-4537-4fda-9321-b930d128649c/volumes"
Jul 10 14:37:18.461887 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:18.461855    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:19.947489 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:19.947456    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:25.740712 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:25.740672    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:25.851094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:25.851060    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:26.023427 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:26.023347    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:26.304483 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:26.304395    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:27.389062 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:27.389028    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:27.490855 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:27.490798    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:27.589826 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:27.589793    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:28.359058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:28.359024    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:28.458494 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:28.458463    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:28.559910 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:28.559873    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:30.022977 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:30.022940    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:36.328425 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:36.328391    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:36.431928 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:36.431890    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:37.019031 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:37.018997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:43.621034 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:43.620999    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:43.729624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:43.729593    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:43.826822 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:43.826790    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:54.397043 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:54.397011    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:37:54.748180 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:37:54.748090    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:15.736751 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:15.736701    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:21.632668 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:21.632632    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:21.720115 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:21.720069    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:22.032766 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:22.032659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:22.477347 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:22.477311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:22.577474 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:22.577443    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:25.720022 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.719989    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:25.863736 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.863691    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:25.896522 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.896489    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"]
Jul 10 14:38:25.896852 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.896841    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c1e0f4e-4537-4fda-9321-b930d128649c" containerName="main"
Jul 10 14:38:25.896903 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.896854    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c1e0f4e-4537-4fda-9321-b930d128649c" containerName="main"
Jul 10 14:38:25.896903 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.896880    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c374f93-c084-4cff-9244-22ea8239ee34" containerName="maas-api"
Jul 10 14:38:25.896903 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.896885    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c374f93-c084-4cff-9244-22ea8239ee34" containerName="maas-api"
Jul 10 14:38:25.896993 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.896948    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c374f93-c084-4cff-9244-22ea8239ee34" containerName="maas-api"
Jul 10 14:38:25.896993 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.896957    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c1e0f4e-4537-4fda-9321-b930d128649c" containerName="main"
Jul 10 14:38:25.899650 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.899634    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:25.904188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.904167    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ce362d-serving-cert\""
Jul 10 14:38:25.955104 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.955069    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk5wb\" (UniqueName: \"kubernetes.io/projected/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-kube-api-access-vk5wb\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:25.955271 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.955115    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:25.955271 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:25.955205    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.016663 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.016582    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:26.029977 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.029947    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"]
Jul 10 14:38:26.056435 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.056411    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk5wb\" (UniqueName: \"kubernetes.io/projected/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-kube-api-access-vk5wb\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.056571 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.056446    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.056571 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.056490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.056648 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:38:26.056607    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-serving-cert: secret "maas-api-e2e-inf-a-ce362d-serving-cert" not found
Jul 10 14:38:26.056682 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:38:26.056677    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls podName:74aba9b9-4506-45aa-9169-0e2cbc6c5f37 nodeName:}" failed. No retries permitted until 2026-07-10 14:38:26.556657637 +0000 UTC m=+1778.182383554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls") pod "maas-api-e2e-inf-a-ce362d-565bf95798-nst7r" (UID: "74aba9b9-4506-45aa-9169-0e2cbc6c5f37") : secret "maas-api-e2e-inf-a-ce362d-serving-cert" not found
Jul 10 14:38:26.057055 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.057037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.080551 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.080533    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk5wb\" (UniqueName: \"kubernetes.io/projected/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-kube-api-access-vk5wb\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.356146 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.356113    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:26.562000 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.561969    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.564341 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.564316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce362d-565bf95798-nst7r\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.808984 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.808903    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:26.828606 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.828577    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:26.929954 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.929927    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:26.950477 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.950452    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"]
Jul 10 14:38:26.951768 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:38:26.951740    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74aba9b9_4506_45aa_9169_0e2cbc6c5f37.slice/crio-26be23ae5472d11f8637d2516f4dc120ce406dc2410284b5aa287d8328f96459 WatchSource:0}: Error finding container 26be23ae5472d11f8637d2516f4dc120ce406dc2410284b5aa287d8328f96459: Status 404 returned error can't find the container with id 26be23ae5472d11f8637d2516f4dc120ce406dc2410284b5aa287d8328f96459
Jul 10 14:38:26.952997 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:26.952979    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:38:27.941482 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:27.941394    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r" event={"ID":"74aba9b9-4506-45aa-9169-0e2cbc6c5f37","Type":"ContainerStarted","Data":"30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991"}
Jul 10 14:38:27.941482 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:27.941455    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r" event={"ID":"74aba9b9-4506-45aa-9169-0e2cbc6c5f37","Type":"ContainerStarted","Data":"26be23ae5472d11f8637d2516f4dc120ce406dc2410284b5aa287d8328f96459"}
Jul 10 14:38:27.941914 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:27.941536    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:27.965328 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:27.965281    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r" podStartSLOduration=2.266653272 podStartE2EDuration="2.965265586s" totalImagesPullingTime="698.612314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:38:27.963547194 +0000 UTC m=+1779.589273126" watchObservedRunningTime="2026-07-10 14:38:27.965265586 +0000 UTC m=+1779.590991523"
Jul 10 14:38:30.213065 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:30.213033    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:30.592038 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:30.592008    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:30.940243 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:30.940168    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:31.030800 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:31.030770    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:33.672250 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:33.672217    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:33.849674 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:33.849644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:33.917663 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:33.917639    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"]
Jul 10 14:38:33.921102 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:33.921081    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:33.925010 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:33.924951    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a8bf33-serving-cert\""
Jul 10 14:38:33.952250 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:33.952234    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:38:33.955900 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:33.955880    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"]
Jul 10 14:38:34.007549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.007509    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:34.026645 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.026621    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.026796 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.026697    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq76w\" (UniqueName: \"kubernetes.io/projected/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-kube-api-access-rq76w\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.026871 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.026849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.127714 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.127689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.127887 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.127767    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq76w\" (UniqueName: \"kubernetes.io/projected/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-kube-api-access-rq76w\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.127887 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.127829    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.128004 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:38:34.127922    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-serving-cert: secret "maas-api-e2e-inf-b-a8bf33-serving-cert" not found
Jul 10 14:38:34.128004 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:38:34.127975    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls podName:e0e8e673-224c-4304-aa1d-2e72c2ec9a69 nodeName:}" failed. No retries permitted until 2026-07-10 14:38:34.627957888 +0000 UTC m=+1786.253683797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls") pod "maas-api-e2e-inf-b-a8bf33-977598d87-2pflv" (UID: "e0e8e673-224c-4304-aa1d-2e72c2ec9a69") : secret "maas-api-e2e-inf-b-a8bf33-serving-cert" not found
Jul 10 14:38:34.128327 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.128307    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.139039 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.139014    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq76w\" (UniqueName: \"kubernetes.io/projected/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-kube-api-access-rq76w\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.447254 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.447220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:34.605968 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.605937    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:34.632218 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.632187    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.634476 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.634449    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls\") pod \"maas-api-e2e-inf-b-a8bf33-977598d87-2pflv\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.706183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.706109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:34.831394 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.831358    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:34.968823 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:34.968797    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"]
Jul 10 14:38:34.971474 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:38:34.971443    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e8e673_224c_4304_aa1d_2e72c2ec9a69.slice/crio-56c7e3ad714a9520d43b7c7ec2eb6b2dc99014c327928aaf1daea3e613a00491 WatchSource:0}: Error finding container 56c7e3ad714a9520d43b7c7ec2eb6b2dc99014c327928aaf1daea3e613a00491: Status 404 returned error can't find the container with id 56c7e3ad714a9520d43b7c7ec2eb6b2dc99014c327928aaf1daea3e613a00491
Jul 10 14:38:35.980008 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:35.979927    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv" event={"ID":"e0e8e673-224c-4304-aa1d-2e72c2ec9a69","Type":"ContainerStarted","Data":"3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8"}
Jul 10 14:38:35.980008 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:35.979976    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv" event={"ID":"e0e8e673-224c-4304-aa1d-2e72c2ec9a69","Type":"ContainerStarted","Data":"56c7e3ad714a9520d43b7c7ec2eb6b2dc99014c327928aaf1daea3e613a00491"}
Jul 10 14:38:35.980386 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:35.980074    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:36.000608 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:36.000569    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv" podStartSLOduration=2.364765159 podStartE2EDuration="3.000553502s" totalImagesPullingTime="635.788343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:38:35.998936454 +0000 UTC m=+1787.624662368" watchObservedRunningTime="2026-07-10 14:38:36.000553502 +0000 UTC m=+1787.626279433"
Jul 10 14:38:38.253180 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:38.253148    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:41.989651 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:41.989583    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:38:47.288853 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.288819    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"]
Jul 10 14:38:47.292328 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.292310    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.298497 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.298470    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ce362d\"/\"openshift-service-ca.crt\""
Jul 10 14:38:47.298497 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.298496    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ce362d\"/\"kube-root-ca.crt\""
Jul 10 14:38:47.298688 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.298509    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ce362d\"/\"default-dockercfg-nvltg\""
Jul 10 14:38:47.298688 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.298555    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ce362d\"/\"test-model-ce362d-kserve-self-signed-certs\""
Jul 10 14:38:47.317485 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.317463    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"]
Jul 10 14:38:47.330613 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.330589    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:47.448532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.448497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r4p2\" (UniqueName: \"kubernetes.io/projected/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-kube-api-access-2r4p2\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.448657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.448551    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-model-cache\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.448657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.448579    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-dshm\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.448657 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.448641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-home\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.448838 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.448663    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tls-certs\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.448838 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.448696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tmp-dir\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.549713 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.549646    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tmp-dir\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.549713 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.549692    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r4p2\" (UniqueName: \"kubernetes.io/projected/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-kube-api-access-2r4p2\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.549924 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.549742    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-model-cache\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.549924 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.549772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-dshm\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.550034 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.549926    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-home\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.550034 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.549962    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tls-certs\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.550177 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.550156    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tmp-dir\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.550244 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.550216    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-model-cache\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.550295 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.550277    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-home\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.552038 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.552016    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-dshm\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.552349 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.552333    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tls-certs\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.559010 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.558988    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r4p2\" (UniqueName: \"kubernetes.io/projected/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-kube-api-access-2r4p2\") pod \"test-model-ce362d-kserve-ddf5798c9-gmwt7\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") " pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.601535 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.601509    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:47.735998 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:47.735973    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"]
Jul 10 14:38:47.738224 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:38:47.738187    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91a4ff6b_127b_4e4b_976e_8fa22b7e769f.slice/crio-e8d5fbbd3bf1e1c633f8d1a1a3f86fd0b8e2525fb8b8d69e9db7c91403fb65ee WatchSource:0}: Error finding container e8d5fbbd3bf1e1c633f8d1a1a3f86fd0b8e2525fb8b8d69e9db7c91403fb65ee: Status 404 returned error can't find the container with id e8d5fbbd3bf1e1c633f8d1a1a3f86fd0b8e2525fb8b8d69e9db7c91403fb65ee
Jul 10 14:38:48.035859 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:48.035826    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7" event={"ID":"91a4ff6b-127b-4e4b-976e-8fa22b7e769f","Type":"ContainerStarted","Data":"19e3df09e8393655f583553009cff36713c159abac3540df787e29c26f21f170"}
Jul 10 14:38:48.035859 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:48.035865    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7" event={"ID":"91a4ff6b-127b-4e4b-976e-8fa22b7e769f","Type":"ContainerStarted","Data":"e8d5fbbd3bf1e1c633f8d1a1a3f86fd0b8e2525fb8b8d69e9db7c91403fb65ee"}
Jul 10 14:38:48.054524 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:48.054417    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7" podStartSLOduration=1.054403313 podStartE2EDuration="1.054403313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:38:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:38:48.053563755 +0000 UTC m=+1799.679289685" watchObservedRunningTime="2026-07-10 14:38:48.054403313 +0000 UTC m=+1799.680129242"
Jul 10 14:38:48.892996 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:48.892964    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:38:48.895538 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:48.895520    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:38:48.914566 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:48.914543    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:49.017804 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:49.017766    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:55.459084 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:55.459048    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:55.558455 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:55.558427    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:38:57.602428 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:57.602396    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:57.602857 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:57.602471    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:57.614649 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:57.614627    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:38:58.089898 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:38:58.089873    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:39:19.770432 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.770401    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"]
Jul 10 14:39:19.774305 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.774290    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.777458 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.777433    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a8bf33\"/\"openshift-service-ca.crt\""
Jul 10 14:39:19.778866 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.778845    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a8bf33\"/\"kube-root-ca.crt\""
Jul 10 14:39:19.778985 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.778886    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a8bf33\"/\"default-dockercfg-gkp9t\""
Jul 10 14:39:19.778985 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.778925    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a8bf33\"/\"test-model-a8bf33-kserve-self-signed-certs\""
Jul 10 14:39:19.784044 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.784025    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"]
Jul 10 14:39:19.808869 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.808844    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:19.821992 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.821965    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794216fa-12de-4dc8-9a9e-bb87cc63f709-tls-certs\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.822094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.821997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-dshm\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.822094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.822026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-model-cache\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.822188 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.822088    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-tmp-dir\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.822230 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.822192    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbv84\" (UniqueName: \"kubernetes.io/projected/794216fa-12de-4dc8-9a9e-bb87cc63f709-kube-api-access-sbv84\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.822277 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.822253    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-home\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.922639 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.922594    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-home\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.922854 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.922664    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794216fa-12de-4dc8-9a9e-bb87cc63f709-tls-certs\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.922854 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.922686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-dshm\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.922854 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.922703    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-model-cache\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.922854 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.922730    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-tmp-dir\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.922854 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.922758    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbv84\" (UniqueName: \"kubernetes.io/projected/794216fa-12de-4dc8-9a9e-bb87cc63f709-kube-api-access-sbv84\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.923130 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.923055    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-home\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.923190 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.923171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-model-cache\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.923226 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.923169    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-tmp-dir\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.924981 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.924953    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-dshm\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.925223 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.925208    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794216fa-12de-4dc8-9a9e-bb87cc63f709-tls-certs\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:19.931605 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:19.931578    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbv84\" (UniqueName: \"kubernetes.io/projected/794216fa-12de-4dc8-9a9e-bb87cc63f709-kube-api-access-sbv84\") pod \"test-model-a8bf33-kserve-cfcd55f9-xphhx\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") " pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:20.084175 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:20.084146    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:20.216598 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:20.216573    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"]
Jul 10 14:39:20.218422 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:39:20.218396    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod794216fa_12de_4dc8_9a9e_bb87cc63f709.slice/crio-1ccec8e965f7b8ce7c4bf4dcd1983f8f1249648aa99cf3094338ec55a931a249 WatchSource:0}: Error finding container 1ccec8e965f7b8ce7c4bf4dcd1983f8f1249648aa99cf3094338ec55a931a249: Status 404 returned error can't find the container with id 1ccec8e965f7b8ce7c4bf4dcd1983f8f1249648aa99cf3094338ec55a931a249
Jul 10 14:39:21.189776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:21.189738    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx" event={"ID":"794216fa-12de-4dc8-9a9e-bb87cc63f709","Type":"ContainerStarted","Data":"6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38"}
Jul 10 14:39:21.190226 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:21.189792    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx" event={"ID":"794216fa-12de-4dc8-9a9e-bb87cc63f709","Type":"ContainerStarted","Data":"1ccec8e965f7b8ce7c4bf4dcd1983f8f1249648aa99cf3094338ec55a931a249"}
Jul 10 14:39:21.210648 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:21.210595    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx" podStartSLOduration=2.210578384 podStartE2EDuration="2.210578384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:39:21.208711638 +0000 UTC m=+1832.834437570" watchObservedRunningTime="2026-07-10 14:39:21.210578384 +0000 UTC m=+1832.836304322"
Jul 10 14:39:30.085039 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:30.085004    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:30.085549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:30.085239    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:30.097478 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:30.097458    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:30.242647 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:30.242616    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:37.079416 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:37.079380    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"]
Jul 10 14:39:37.100082 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:37.100049    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:37.952126 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:37.952092    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"]
Jul 10 14:39:38.609784 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:38.609754    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:38.930219 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:38.930132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:39.030398 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.030368    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:39.213199 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.213125    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"]
Jul 10 14:39:39.213345 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.213311    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r" podUID="74aba9b9-4506-45aa-9169-0e2cbc6c5f37" containerName="maas-api" containerID="cri-o://30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991" gracePeriod=30
Jul 10 14:39:39.243271 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.243249    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:39.452951 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.452926    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:39:39.601758 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.601714    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk5wb\" (UniqueName: \"kubernetes.io/projected/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-kube-api-access-vk5wb\") pod \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") "
Jul 10 14:39:39.601937 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.601828    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls\") pod \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") "
Jul 10 14:39:39.601937 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.601855    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-openshift-service-ca\") pod \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\" (UID: \"74aba9b9-4506-45aa-9169-0e2cbc6c5f37\") "
Jul 10 14:39:39.602238 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.602204    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74aba9b9-4506-45aa-9169-0e2cbc6c5f37" (UID: "74aba9b9-4506-45aa-9169-0e2cbc6c5f37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:39:39.603794 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.603770    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-kube-api-access-vk5wb" (OuterVolumeSpecName: "kube-api-access-vk5wb") pod "74aba9b9-4506-45aa-9169-0e2cbc6c5f37" (UID: "74aba9b9-4506-45aa-9169-0e2cbc6c5f37"). InnerVolumeSpecName "kube-api-access-vk5wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:39:39.603899 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.603879    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74aba9b9-4506-45aa-9169-0e2cbc6c5f37" (UID: "74aba9b9-4506-45aa-9169-0e2cbc6c5f37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:39:39.702771 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.702745    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk5wb\" (UniqueName: \"kubernetes.io/projected/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-kube-api-access-vk5wb\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:39.702771 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.702767    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:39.702771 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:39.702777    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74aba9b9-4506-45aa-9169-0e2cbc6c5f37-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:40.277694 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.277656    2584 generic.go:356] "Generic (PLEG): container finished" podID="74aba9b9-4506-45aa-9169-0e2cbc6c5f37" containerID="30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991" exitCode=0
Jul 10 14:39:40.277894 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.277705    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"
Jul 10 14:39:40.277894 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.277739    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r" event={"ID":"74aba9b9-4506-45aa-9169-0e2cbc6c5f37","Type":"ContainerDied","Data":"30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991"}
Jul 10 14:39:40.277894 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.277794    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r" event={"ID":"74aba9b9-4506-45aa-9169-0e2cbc6c5f37","Type":"ContainerDied","Data":"26be23ae5472d11f8637d2516f4dc120ce406dc2410284b5aa287d8328f96459"}
Jul 10 14:39:40.277894 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.277818    2584 scope.go:117] "RemoveContainer" containerID="30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991"
Jul 10 14:39:40.290861 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.290603    2584 scope.go:117] "RemoveContainer" containerID="30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991"
Jul 10 14:39:40.290940 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:39:40.290924    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991\": container with ID starting with 30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991 not found: ID does not exist" containerID="30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991"
Jul 10 14:39:40.290988 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.290956    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991"} err="failed to get container status \"30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991\": rpc error: code = NotFound desc = could not find container \"30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991\": container with ID starting with 30ebb5553177b76baaea6cac0c458cd0a35e78148f56636c53a9cfc1c94a1991 not found: ID does not exist"
Jul 10 14:39:40.306300 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.306274    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"]
Jul 10 14:39:40.318424 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.318400    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce362d-565bf95798-nst7r"]
Jul 10 14:39:40.921626 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:40.921592    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74aba9b9-4506-45aa-9169-0e2cbc6c5f37" path="/var/lib/kubelet/pods/74aba9b9-4506-45aa-9169-0e2cbc6c5f37/volumes"
Jul 10 14:39:43.658518 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:43.658484    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:39:52.099193 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.099153    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7" podUID="91a4ff6b-127b-4e4b-976e-8fa22b7e769f" containerName="main" containerID="cri-o://19e3df09e8393655f583553009cff36713c159abac3540df787e29c26f21f170" gracePeriod=45
Jul 10 14:39:52.332892 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.332866    2584 generic.go:356] "Generic (PLEG): container finished" podID="91a4ff6b-127b-4e4b-976e-8fa22b7e769f" containerID="19e3df09e8393655f583553009cff36713c159abac3540df787e29c26f21f170" exitCode=0
Jul 10 14:39:52.333009 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.332902    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7" event={"ID":"91a4ff6b-127b-4e4b-976e-8fa22b7e769f","Type":"ContainerDied","Data":"19e3df09e8393655f583553009cff36713c159abac3540df787e29c26f21f170"}
Jul 10 14:39:52.350652 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.350593    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:39:52.520142 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520112    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-dshm\") pod \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") "
Jul 10 14:39:52.520322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520171    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-model-cache\") pod \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") "
Jul 10 14:39:52.520322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520220    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tls-certs\") pod \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") "
Jul 10 14:39:52.520322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520293    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-home\") pod \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") "
Jul 10 14:39:52.520485 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520335    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tmp-dir\") pod \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") "
Jul 10 14:39:52.520485 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520379    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r4p2\" (UniqueName: \"kubernetes.io/projected/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-kube-api-access-2r4p2\") pod \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\" (UID: \"91a4ff6b-127b-4e4b-976e-8fa22b7e769f\") "
Jul 10 14:39:52.520586 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520513    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-model-cache" (OuterVolumeSpecName: "model-cache") pod "91a4ff6b-127b-4e4b-976e-8fa22b7e769f" (UID: "91a4ff6b-127b-4e4b-976e-8fa22b7e769f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:52.520586 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520555    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-home" (OuterVolumeSpecName: "home") pod "91a4ff6b-127b-4e4b-976e-8fa22b7e769f" (UID: "91a4ff6b-127b-4e4b-976e-8fa22b7e769f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:52.520792 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520709    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-model-cache\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:52.520792 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520760    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-home\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:52.520792 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.520712    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "91a4ff6b-127b-4e4b-976e-8fa22b7e769f" (UID: "91a4ff6b-127b-4e4b-976e-8fa22b7e769f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:52.522988 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.522944    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-dshm" (OuterVolumeSpecName: "dshm") pod "91a4ff6b-127b-4e4b-976e-8fa22b7e769f" (UID: "91a4ff6b-127b-4e4b-976e-8fa22b7e769f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:52.523115 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.523010    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-kube-api-access-2r4p2" (OuterVolumeSpecName: "kube-api-access-2r4p2") pod "91a4ff6b-127b-4e4b-976e-8fa22b7e769f" (UID: "91a4ff6b-127b-4e4b-976e-8fa22b7e769f"). InnerVolumeSpecName "kube-api-access-2r4p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:39:52.523115 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.523068    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "91a4ff6b-127b-4e4b-976e-8fa22b7e769f" (UID: "91a4ff6b-127b-4e4b-976e-8fa22b7e769f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:39:52.621945 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.621869    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r4p2\" (UniqueName: \"kubernetes.io/projected/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-kube-api-access-2r4p2\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:52.621945 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.621894    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-dshm\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:52.621945 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.621905    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tls-certs\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:52.621945 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.621914    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a4ff6b-127b-4e4b-976e-8fa22b7e769f-tmp-dir\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:52.972682 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:52.972615    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx" podUID="794216fa-12de-4dc8-9a9e-bb87cc63f709" containerName="main" containerID="cri-o://6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38" gracePeriod=45
Jul 10 14:39:53.217457 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.217430    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:53.328494 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328459    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-model-cache\") pod \"794216fa-12de-4dc8-9a9e-bb87cc63f709\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") "
Jul 10 14:39:53.328683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328515    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-tmp-dir\") pod \"794216fa-12de-4dc8-9a9e-bb87cc63f709\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") "
Jul 10 14:39:53.328683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328564    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-dshm\") pod \"794216fa-12de-4dc8-9a9e-bb87cc63f709\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") "
Jul 10 14:39:53.328683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328592    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-home\") pod \"794216fa-12de-4dc8-9a9e-bb87cc63f709\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") "
Jul 10 14:39:53.328683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328617    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794216fa-12de-4dc8-9a9e-bb87cc63f709-tls-certs\") pod \"794216fa-12de-4dc8-9a9e-bb87cc63f709\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") "
Jul 10 14:39:53.328683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328655    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbv84\" (UniqueName: \"kubernetes.io/projected/794216fa-12de-4dc8-9a9e-bb87cc63f709-kube-api-access-sbv84\") pod \"794216fa-12de-4dc8-9a9e-bb87cc63f709\" (UID: \"794216fa-12de-4dc8-9a9e-bb87cc63f709\") "
Jul 10 14:39:53.328964 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328806    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-model-cache" (OuterVolumeSpecName: "model-cache") pod "794216fa-12de-4dc8-9a9e-bb87cc63f709" (UID: "794216fa-12de-4dc8-9a9e-bb87cc63f709"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:53.328964 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328925    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "794216fa-12de-4dc8-9a9e-bb87cc63f709" (UID: "794216fa-12de-4dc8-9a9e-bb87cc63f709"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:53.329076 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.328985    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-model-cache\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:53.329076 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.329065    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-home" (OuterVolumeSpecName: "home") pod "794216fa-12de-4dc8-9a9e-bb87cc63f709" (UID: "794216fa-12de-4dc8-9a9e-bb87cc63f709"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:53.330870 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.330846    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794216fa-12de-4dc8-9a9e-bb87cc63f709-kube-api-access-sbv84" (OuterVolumeSpecName: "kube-api-access-sbv84") pod "794216fa-12de-4dc8-9a9e-bb87cc63f709" (UID: "794216fa-12de-4dc8-9a9e-bb87cc63f709"). InnerVolumeSpecName "kube-api-access-sbv84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:39:53.331088 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.331070    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-dshm" (OuterVolumeSpecName: "dshm") pod "794216fa-12de-4dc8-9a9e-bb87cc63f709" (UID: "794216fa-12de-4dc8-9a9e-bb87cc63f709"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:39:53.331165 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.331068    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/794216fa-12de-4dc8-9a9e-bb87cc63f709-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "794216fa-12de-4dc8-9a9e-bb87cc63f709" (UID: "794216fa-12de-4dc8-9a9e-bb87cc63f709"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:39:53.339033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.339011    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"
Jul 10 14:39:53.339033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.339015    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7" event={"ID":"91a4ff6b-127b-4e4b-976e-8fa22b7e769f","Type":"ContainerDied","Data":"e8d5fbbd3bf1e1c633f8d1a1a3f86fd0b8e2525fb8b8d69e9db7c91403fb65ee"}
Jul 10 14:39:53.339206 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.339072    2584 scope.go:117] "RemoveContainer" containerID="19e3df09e8393655f583553009cff36713c159abac3540df787e29c26f21f170"
Jul 10 14:39:53.340463 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.340444    2584 generic.go:356] "Generic (PLEG): container finished" podID="794216fa-12de-4dc8-9a9e-bb87cc63f709" containerID="6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38" exitCode=0
Jul 10 14:39:53.340536 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.340477    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"
Jul 10 14:39:53.340591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.340526    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx" event={"ID":"794216fa-12de-4dc8-9a9e-bb87cc63f709","Type":"ContainerDied","Data":"6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38"}
Jul 10 14:39:53.340591 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.340573    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx" event={"ID":"794216fa-12de-4dc8-9a9e-bb87cc63f709","Type":"ContainerDied","Data":"1ccec8e965f7b8ce7c4bf4dcd1983f8f1249648aa99cf3094338ec55a931a249"}
Jul 10 14:39:53.353319 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.353302    2584 scope.go:117] "RemoveContainer" containerID="6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38"
Jul 10 14:39:53.360832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.360758    2584 scope.go:117] "RemoveContainer" containerID="6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38"
Jul 10 14:39:53.361041 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:39:53.361025    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38\": container with ID starting with 6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38 not found: ID does not exist" containerID="6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38"
Jul 10 14:39:53.361086 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.361050    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38"} err="failed to get container status \"6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38\": rpc error: code = NotFound desc = could not find container \"6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38\": container with ID starting with 6d32a6c9c60a092b655ce17d674b73093a84dd5a42a6c548c813a30ada77ab38 not found: ID does not exist"
Jul 10 14:39:53.365912 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.365891    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"]
Jul 10 14:39:53.370275 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.370251    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-ce362d/test-model-ce362d-kserve-ddf5798c9-gmwt7"]
Jul 10 14:39:53.383207 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.383181    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"]
Jul 10 14:39:53.389292 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.389267    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a8bf33/test-model-a8bf33-kserve-cfcd55f9-xphhx"]
Jul 10 14:39:53.429806 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.429776    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-tmp-dir\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:53.429806 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.429803    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-dshm\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:53.429806 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.429814    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794216fa-12de-4dc8-9a9e-bb87cc63f709-home\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:53.430020 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.429822    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794216fa-12de-4dc8-9a9e-bb87cc63f709-tls-certs\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:53.430020 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:53.429830    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbv84\" (UniqueName: \"kubernetes.io/projected/794216fa-12de-4dc8-9a9e-bb87cc63f709-kube-api-access-sbv84\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:39:54.921901 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:54.921869    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="794216fa-12de-4dc8-9a9e-bb87cc63f709" path="/var/lib/kubelet/pods/794216fa-12de-4dc8-9a9e-bb87cc63f709/volumes"
Jul 10 14:39:54.922266 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:39:54.922227    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91a4ff6b-127b-4e4b-976e-8fa22b7e769f" path="/var/lib/kubelet/pods/91a4ff6b-127b-4e4b-976e-8fa22b7e769f/volumes"
Jul 10 14:40:15.352607 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:15.352512    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:15.452827 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:15.452794    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:20.915587 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:20.915537    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:21.196368 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:21.196276    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:21.465888 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:21.465796    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:21.658654 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:21.658624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:22.152953 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.152922    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:22.277816 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.277782    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:22.374893 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.374855    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:22.566646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.566618    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"]
Jul 10 14:40:22.566889 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.566865    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv" podUID="e0e8e673-224c-4304-aa1d-2e72c2ec9a69" containerName="maas-api" containerID="cri-o://3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8" gracePeriod=30
Jul 10 14:40:22.604319 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.604291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:22.813058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.813037    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:40:22.870907 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.870855    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls\") pod \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") "
Jul 10 14:40:22.871028 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.870924    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq76w\" (UniqueName: \"kubernetes.io/projected/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-kube-api-access-rq76w\") pod \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") "
Jul 10 14:40:22.871028 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.870977    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-openshift-service-ca\") pod \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\" (UID: \"e0e8e673-224c-4304-aa1d-2e72c2ec9a69\") "
Jul 10 14:40:22.871432 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.871402    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0e8e673-224c-4304-aa1d-2e72c2ec9a69" (UID: "e0e8e673-224c-4304-aa1d-2e72c2ec9a69"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:40:22.872779 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.872757    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0e8e673-224c-4304-aa1d-2e72c2ec9a69" (UID: "e0e8e673-224c-4304-aa1d-2e72c2ec9a69"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:40:22.872945 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.872927    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-kube-api-access-rq76w" (OuterVolumeSpecName: "kube-api-access-rq76w") pod "e0e8e673-224c-4304-aa1d-2e72c2ec9a69" (UID: "e0e8e673-224c-4304-aa1d-2e72c2ec9a69"). InnerVolumeSpecName "kube-api-access-rq76w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:40:22.972348 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.972321    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:40:22.972348 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.972346    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq76w\" (UniqueName: \"kubernetes.io/projected/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-kube-api-access-rq76w\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:40:22.972513 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:22.972356    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e8e673-224c-4304-aa1d-2e72c2ec9a69-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:40:23.477008 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.476973    2584 generic.go:356] "Generic (PLEG): container finished" podID="e0e8e673-224c-4304-aa1d-2e72c2ec9a69" containerID="3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8" exitCode=0
Jul 10 14:40:23.477433 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.477017    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"
Jul 10 14:40:23.477433 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.477025    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv" event={"ID":"e0e8e673-224c-4304-aa1d-2e72c2ec9a69","Type":"ContainerDied","Data":"3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8"}
Jul 10 14:40:23.477433 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.477066    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv" event={"ID":"e0e8e673-224c-4304-aa1d-2e72c2ec9a69","Type":"ContainerDied","Data":"56c7e3ad714a9520d43b7c7ec2eb6b2dc99014c327928aaf1daea3e613a00491"}
Jul 10 14:40:23.477433 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.477089    2584 scope.go:117] "RemoveContainer" containerID="3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8"
Jul 10 14:40:23.485756 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.485733    2584 scope.go:117] "RemoveContainer" containerID="3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8"
Jul 10 14:40:23.486100 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:40:23.486045    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8\": container with ID starting with 3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8 not found: ID does not exist" containerID="3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8"
Jul 10 14:40:23.486100 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.486084    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8"} err="failed to get container status \"3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8\": rpc error: code = NotFound desc = could not find container \"3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8\": container with ID starting with 3c18aa952710f564d2721882f62e760c758d55b25ec228c9efe83f89470413c8 not found: ID does not exist"
Jul 10 14:40:23.497646 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.497612    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"]
Jul 10 14:40:23.501625 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:23.501599    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a8bf33-977598d87-2pflv"]
Jul 10 14:40:24.921997 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:24.921959    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0e8e673-224c-4304-aa1d-2e72c2ec9a69" path="/var/lib/kubelet/pods/e0e8e673-224c-4304-aa1d-2e72c2ec9a69/volumes"
Jul 10 14:40:28.692047 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:28.692011    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:28.883962 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:28.883927    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:29.028069 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:29.027987    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:29.189971 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:29.189937    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:29.569823 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:29.569792    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:29.850832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:29.850753    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:30.557260 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:30.557226    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:33.230700 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.230670    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:33.326649 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.326603    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:33.363238 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363203    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"]
Jul 10 14:40:33.363538 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363526    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a4ff6b-127b-4e4b-976e-8fa22b7e769f" containerName="main"
Jul 10 14:40:33.363581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363540    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a4ff6b-127b-4e4b-976e-8fa22b7e769f" containerName="main"
Jul 10 14:40:33.363581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363551    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74aba9b9-4506-45aa-9169-0e2cbc6c5f37" containerName="maas-api"
Jul 10 14:40:33.363581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363557    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="74aba9b9-4506-45aa-9169-0e2cbc6c5f37" containerName="maas-api"
Jul 10 14:40:33.363581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363574    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e8e673-224c-4304-aa1d-2e72c2ec9a69" containerName="maas-api"
Jul 10 14:40:33.363581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363580    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e8e673-224c-4304-aa1d-2e72c2ec9a69" containerName="maas-api"
Jul 10 14:40:33.363757 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363588    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794216fa-12de-4dc8-9a9e-bb87cc63f709" containerName="main"
Jul 10 14:40:33.363757 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363594    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="794216fa-12de-4dc8-9a9e-bb87cc63f709" containerName="main"
Jul 10 14:40:33.363757 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363642    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="74aba9b9-4506-45aa-9169-0e2cbc6c5f37" containerName="maas-api"
Jul 10 14:40:33.363757 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363650    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="794216fa-12de-4dc8-9a9e-bb87cc63f709" containerName="main"
Jul 10 14:40:33.363757 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363658    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e8e673-224c-4304-aa1d-2e72c2ec9a69" containerName="maas-api"
Jul 10 14:40:33.363757 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.363664    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="91a4ff6b-127b-4e4b-976e-8fa22b7e769f" containerName="main"
Jul 10 14:40:33.367959 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.367942    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.371004 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.370979    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5757a4-serving-cert\""
Jul 10 14:40:33.378624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.378603    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"]
Jul 10 14:40:33.462750 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.462696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-openshift-service-ca\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.462917 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.462806    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.462917 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.462848    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb25z\" (UniqueName: \"kubernetes.io/projected/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-kube-api-access-jb25z\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.519876 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.519805    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:33.563900 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.563871    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.564016 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.563913    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb25z\" (UniqueName: \"kubernetes.io/projected/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-kube-api-access-jb25z\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.564016 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.563968    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-openshift-service-ca\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.564108 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:40:33.564022    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-serving-cert: secret "maas-api-e2e-api-a-5757a4-serving-cert" not found
Jul 10 14:40:33.564108 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:40:33.564102    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls podName:ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da nodeName:}" failed. No retries permitted until 2026-07-10 14:40:34.064082833 +0000 UTC m=+1905.689808743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls") pod "maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw" (UID: "ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da") : secret "maas-api-e2e-api-a-5757a4-serving-cert" not found
Jul 10 14:40:33.564479 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.564463    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-openshift-service-ca\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.583490 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.583462    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb25z\" (UniqueName: \"kubernetes.io/projected/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-kube-api-access-jb25z\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:33.669329 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:33.669300    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:34.066898 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:34.066863    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:34.069294 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:34.069263    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls\") pod \"maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:34.121577 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:34.121531    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:34.236992 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:34.236953    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:34.278220 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:34.278190    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:34.437160 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:34.437138    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"]
Jul 10 14:40:34.439186 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:40:34.439159    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1e1c13_ad6c_4e52_8a0d_349ab53eb4da.slice/crio-0bf4f2dd2b774a35c7128cc67637d6a26d4405af7f5200dc2af8ee465622610f WatchSource:0}: Error finding container 0bf4f2dd2b774a35c7128cc67637d6a26d4405af7f5200dc2af8ee465622610f: Status 404 returned error can't find the container with id 0bf4f2dd2b774a35c7128cc67637d6a26d4405af7f5200dc2af8ee465622610f
Jul 10 14:40:34.536575 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:34.536539    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw" event={"ID":"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da","Type":"ContainerStarted","Data":"0bf4f2dd2b774a35c7128cc67637d6a26d4405af7f5200dc2af8ee465622610f"}
Jul 10 14:40:35.542524 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:35.542484    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw" event={"ID":"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da","Type":"ContainerStarted","Data":"9be81de2a16268b8dfc12ff294272902312ea660856ac5979fe7cdbd370cea0e"}
Jul 10 14:40:35.542969 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:35.542594    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:35.564219 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:35.564174    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw" podStartSLOduration=2.197651173 podStartE2EDuration="2.564159729s" totalImagesPullingTime="366.508556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:35.561400151 +0000 UTC m=+1907.187126085" watchObservedRunningTime="2026-07-10 14:40:35.564159729 +0000 UTC m=+1907.189885660"
Jul 10 14:40:37.708598 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:37.708561    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:38.189045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:38.189009    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:38.524090 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:38.523997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:38.862907 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:38.862861    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:41.552314 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:41.552284    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:43.746322 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.746270    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:43.824297 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.824265    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"]
Jul 10 14:40:43.828094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.828074    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:43.833750 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.833706    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8bc503-serving-cert\""
Jul 10 14:40:43.853744 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.853696    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"]
Jul 10 14:40:43.939176 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.939137    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:43.964965 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.964930    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:43.965135 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.964971    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b32d499-2599-457c-8579-860aa5f22e01-openshift-service-ca\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:43.965135 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:43.964995    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9fm4\" (UniqueName: \"kubernetes.io/projected/4b32d499-2599-457c-8579-860aa5f22e01-kube-api-access-d9fm4\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.065783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.065746    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.065976 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.065800    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b32d499-2599-457c-8579-860aa5f22e01-openshift-service-ca\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.065976 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.065833    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9fm4\" (UniqueName: \"kubernetes.io/projected/4b32d499-2599-457c-8579-860aa5f22e01-kube-api-access-d9fm4\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.065976 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:40:44.065844    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-serving-cert: secret "maas-api-e2e-api-b-8bc503-serving-cert" not found
Jul 10 14:40:44.065976 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:40:44.065936    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls podName:4b32d499-2599-457c-8579-860aa5f22e01 nodeName:}" failed. No retries permitted until 2026-07-10 14:40:44.565913009 +0000 UTC m=+1916.191638919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls") pod "maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf" (UID: "4b32d499-2599-457c-8579-860aa5f22e01") : secret "maas-api-e2e-api-b-8bc503-serving-cert" not found
Jul 10 14:40:44.066579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.066556    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b32d499-2599-457c-8579-860aa5f22e01-openshift-service-ca\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.087624 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.087594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9fm4\" (UniqueName: \"kubernetes.io/projected/4b32d499-2599-457c-8579-860aa5f22e01-kube-api-access-d9fm4\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.440226 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.440141    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:44.570604 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.570567    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.573009 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.572978    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls\") pod \"maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.737460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.737376    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:44.772299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.772237    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:44.897573 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.897549    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"]
Jul 10 14:40:44.900139 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:40:44.900115    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b32d499_2599_457c_8579_860aa5f22e01.slice/crio-2256f04a192bea94f9afc6b5d536be4519056fa57314d90597ba390a24e93bee WatchSource:0}: Error finding container 2256f04a192bea94f9afc6b5d536be4519056fa57314d90597ba390a24e93bee: Status 404 returned error can't find the container with id 2256f04a192bea94f9afc6b5d536be4519056fa57314d90597ba390a24e93bee
Jul 10 14:40:44.930583 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:44.930550    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:45.593912 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:45.593878    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf" event={"ID":"4b32d499-2599-457c-8579-860aa5f22e01","Type":"ContainerStarted","Data":"9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2"}
Jul 10 14:40:45.594144 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:45.593935    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:45.594144 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:45.593952    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf" event={"ID":"4b32d499-2599-457c-8579-860aa5f22e01","Type":"ContainerStarted","Data":"2256f04a192bea94f9afc6b5d536be4519056fa57314d90597ba390a24e93bee"}
Jul 10 14:40:45.615364 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:45.615308    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf" podStartSLOduration=2.226633078 podStartE2EDuration="2.615292614s" totalImagesPullingTime="388.659536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:45.614001503 +0000 UTC m=+1917.239727434" watchObservedRunningTime="2026-07-10 14:40:45.615292614 +0000 UTC m=+1917.241018567"
Jul 10 14:40:48.159170 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:48.159132    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:51.603635 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:51.603601    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:40:56.584201 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:56.584162    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:56.767824 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:56.767786    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:56.899511 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:56.899433    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:57.082146 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:57.082113    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:58.125937 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.125900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:58.327592 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.327546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:58.533040 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.532950    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"]
Jul 10 14:40:58.533204 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.533169    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw" podUID="ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" containerName="maas-api" containerID="cri-o://9be81de2a16268b8dfc12ff294272902312ea660856ac5979fe7cdbd370cea0e" gracePeriod=30
Jul 10 14:40:58.601630 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.601602    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:40:58.657650 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.657622    2584 generic.go:356] "Generic (PLEG): container finished" podID="ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" containerID="9be81de2a16268b8dfc12ff294272902312ea660856ac5979fe7cdbd370cea0e" exitCode=0
Jul 10 14:40:58.657801 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.657697    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw" event={"ID":"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da","Type":"ContainerDied","Data":"9be81de2a16268b8dfc12ff294272902312ea660856ac5979fe7cdbd370cea0e"}
Jul 10 14:40:58.782555 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.782529    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:58.883812 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.883779    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb25z\" (UniqueName: \"kubernetes.io/projected/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-kube-api-access-jb25z\") pod \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") "
Jul 10 14:40:58.883989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.883833    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls\") pod \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") "
Jul 10 14:40:58.883989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.883892    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-openshift-service-ca\") pod \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\" (UID: \"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da\") "
Jul 10 14:40:58.884299 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.884274    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" (UID: "ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:40:58.885905 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.885882    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-kube-api-access-jb25z" (OuterVolumeSpecName: "kube-api-access-jb25z") pod "ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" (UID: "ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da"). InnerVolumeSpecName "kube-api-access-jb25z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:40:58.885985 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.885899    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" (UID: "ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:40:58.984579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.984545    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.984579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.984573    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:40:58.984579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:58.984584    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb25z\" (UniqueName: \"kubernetes.io/projected/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da-kube-api-access-jb25z\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:40:59.663666 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:59.663622    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw" event={"ID":"ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da","Type":"ContainerDied","Data":"0bf4f2dd2b774a35c7128cc67637d6a26d4405af7f5200dc2af8ee465622610f"}
Jul 10 14:40:59.664091 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:59.663691    2584 scope.go:117] "RemoveContainer" containerID="9be81de2a16268b8dfc12ff294272902312ea660856ac5979fe7cdbd370cea0e"
Jul 10 14:40:59.664091 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:59.663638    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"
Jul 10 14:40:59.689156 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:59.689128    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"]
Jul 10 14:40:59.694302 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:40:59.694276    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5757a4-7986ccdcf5-pmtzw"]
Jul 10 14:41:00.921246 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:00.921206    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" path="/var/lib/kubelet/pods/ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da/volumes"
Jul 10 14:41:03.594172 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:03.594124    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:03.711919 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:03.711883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:03.911217 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:03.911134    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:04.110894 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:04.110866    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:04.520674 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:04.520643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:04.905939 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:04.905900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:05.045504 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.045469    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:05.144079 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.144045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:05.301503 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.301425    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"]
Jul 10 14:41:05.301655 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.301631    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf" podUID="4b32d499-2599-457c-8579-860aa5f22e01" containerName="maas-api" containerID="cri-o://9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2" gracePeriod=30
Jul 10 14:41:05.330239 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.330211    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:05.548500 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.548477    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:41:05.643999 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.643971    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls\") pod \"4b32d499-2599-457c-8579-860aa5f22e01\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") "
Jul 10 14:41:05.644160 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.644023    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b32d499-2599-457c-8579-860aa5f22e01-openshift-service-ca\") pod \"4b32d499-2599-457c-8579-860aa5f22e01\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") "
Jul 10 14:41:05.644160 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.644116    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9fm4\" (UniqueName: \"kubernetes.io/projected/4b32d499-2599-457c-8579-860aa5f22e01-kube-api-access-d9fm4\") pod \"4b32d499-2599-457c-8579-860aa5f22e01\" (UID: \"4b32d499-2599-457c-8579-860aa5f22e01\") "
Jul 10 14:41:05.644392 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.644367    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b32d499-2599-457c-8579-860aa5f22e01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b32d499-2599-457c-8579-860aa5f22e01" (UID: "4b32d499-2599-457c-8579-860aa5f22e01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:41:05.646122 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.646091    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b32d499-2599-457c-8579-860aa5f22e01" (UID: "4b32d499-2599-457c-8579-860aa5f22e01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:41:05.646241 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.646124    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b32d499-2599-457c-8579-860aa5f22e01-kube-api-access-d9fm4" (OuterVolumeSpecName: "kube-api-access-d9fm4") pod "4b32d499-2599-457c-8579-860aa5f22e01" (UID: "4b32d499-2599-457c-8579-860aa5f22e01"). InnerVolumeSpecName "kube-api-access-d9fm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:41:05.692827 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.692798    2584 generic.go:356] "Generic (PLEG): container finished" podID="4b32d499-2599-457c-8579-860aa5f22e01" containerID="9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2" exitCode=0
Jul 10 14:41:05.692989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.692851    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"
Jul 10 14:41:05.692989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.692857    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf" event={"ID":"4b32d499-2599-457c-8579-860aa5f22e01","Type":"ContainerDied","Data":"9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2"}
Jul 10 14:41:05.692989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.692963    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf" event={"ID":"4b32d499-2599-457c-8579-860aa5f22e01","Type":"ContainerDied","Data":"2256f04a192bea94f9afc6b5d536be4519056fa57314d90597ba390a24e93bee"}
Jul 10 14:41:05.692989 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.692986    2584 scope.go:117] "RemoveContainer" containerID="9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2"
Jul 10 14:41:05.701713 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.701699    2584 scope.go:117] "RemoveContainer" containerID="9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2"
Jul 10 14:41:05.702079 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:41:05.702060    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2\": container with ID starting with 9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2 not found: ID does not exist" containerID="9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2"
Jul 10 14:41:05.702156 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.702085    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2"} err="failed to get container status \"9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2\": rpc error: code = NotFound desc = could not find container \"9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2\": container with ID starting with 9fddb873313c1050f608775e6eb182386c42fef8d63e51f2e5d14053d4d742c2 not found: ID does not exist"
Jul 10 14:41:05.717683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.717658    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"]
Jul 10 14:41:05.719497 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.719474    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8bc503-5f4c8cd7bd-vnqcf"]
Jul 10 14:41:05.745154 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.745136    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b32d499-2599-457c-8579-860aa5f22e01-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:41:05.745226 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.745156    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b32d499-2599-457c-8579-860aa5f22e01-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:41:05.745226 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:05.745167    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9fm4\" (UniqueName: \"kubernetes.io/projected/4b32d499-2599-457c-8579-860aa5f22e01-kube-api-access-d9fm4\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:41:06.921306 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:06.921268    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b32d499-2599-457c-8579-860aa5f22e01" path="/var/lib/kubelet/pods/4b32d499-2599-457c-8579-860aa5f22e01/volumes"
Jul 10 14:41:10.944257 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:10.944213    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:11.032225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:11.032190    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:11.211331 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:11.211242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:11.432873 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:11.432837    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:11.768910 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:11.768873    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:12.191536 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:12.191501    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:12.618705 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:12.618668    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:15.415220 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.415181    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:15.492426 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.492397    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:15.496241 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.496213    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"]
Jul 10 14:41:15.496553 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.496541    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" containerName="maas-api"
Jul 10 14:41:15.496614 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.496555    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" containerName="maas-api"
Jul 10 14:41:15.496614 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.496565    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b32d499-2599-457c-8579-860aa5f22e01" containerName="maas-api"
Jul 10 14:41:15.496614 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.496571    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b32d499-2599-457c-8579-860aa5f22e01" containerName="maas-api"
Jul 10 14:41:15.496793 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.496620    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b32d499-2599-457c-8579-860aa5f22e01" containerName="maas-api"
Jul 10 14:41:15.496793 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.496630    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab1e1c13-ad6c-4e52-8a0d-349ab53eb4da" containerName="maas-api"
Jul 10 14:41:15.500799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.500780    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.503777 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.503762    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-9e5d79-serving-cert\""
Jul 10 14:41:15.512934 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.512909    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"]
Jul 10 14:41:15.563698 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.563651    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:15.633028 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.632993    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/346ab15b-a527-4c88-a46a-4c3bca94b4df-maas-api-tls\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.633028 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.633026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/346ab15b-a527-4c88-a46a-4c3bca94b4df-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.633259 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.633126    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsghg\" (UniqueName: \"kubernetes.io/projected/346ab15b-a527-4c88-a46a-4c3bca94b4df-kube-api-access-wsghg\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.666671 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.666584    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:15.734297 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.734259    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/346ab15b-a527-4c88-a46a-4c3bca94b4df-maas-api-tls\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.734297 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.734295    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/346ab15b-a527-4c88-a46a-4c3bca94b4df-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.734500 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.734351    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsghg\" (UniqueName: \"kubernetes.io/projected/346ab15b-a527-4c88-a46a-4c3bca94b4df-kube-api-access-wsghg\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.735063 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.735043    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/346ab15b-a527-4c88-a46a-4c3bca94b4df-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.736856 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.736836    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/346ab15b-a527-4c88-a46a-4c3bca94b4df-maas-api-tls\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.745100 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.745078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsghg\" (UniqueName: \"kubernetes.io/projected/346ab15b-a527-4c88-a46a-4c3bca94b4df-kube-api-access-wsghg\") pod \"maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.810539 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.810501    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:15.946307 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:15.946280    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"]
Jul 10 14:41:15.948222 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:41:15.948197    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346ab15b_a527_4c88_a46a_4c3bca94b4df.slice/crio-ec0621e29f9f0f762e5ad8536f3d7415b2deddb8421f5da00400d03183e9bd12 WatchSource:0}: Error finding container ec0621e29f9f0f762e5ad8536f3d7415b2deddb8421f5da00400d03183e9bd12: Status 404 returned error can't find the container with id ec0621e29f9f0f762e5ad8536f3d7415b2deddb8421f5da00400d03183e9bd12
Jul 10 14:41:16.170112 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:16.170082    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:16.269866 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:16.269780    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:16.753872 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:16.753839    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56" event={"ID":"346ab15b-a527-4c88-a46a-4c3bca94b4df","Type":"ContainerStarted","Data":"e0346e24afccec555c23d4b5367692079f7d338a3b26ec750d99ac43eb39685f"}
Jul 10 14:41:16.754225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:16.753890    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56" event={"ID":"346ab15b-a527-4c88-a46a-4c3bca94b4df","Type":"ContainerStarted","Data":"ec0621e29f9f0f762e5ad8536f3d7415b2deddb8421f5da00400d03183e9bd12"}
Jul 10 14:41:16.754225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:16.753915    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:16.775325 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:16.775274    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56" podStartSLOduration=1.315107578 podStartE2EDuration="1.775258157s" totalImagesPullingTime="460.150579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:41:16.77398606 +0000 UTC m=+1948.399711992" watchObservedRunningTime="2026-07-10 14:41:16.775258157 +0000 UTC m=+1948.400984134"
Jul 10 14:41:19.942439 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:19.942402    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:20.292131 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:20.292048    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:20.715479 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:20.715441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:20.825207 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:20.825170    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:22.763447 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:22.763416    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:41:23.838460 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:23.838431    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:23.916482 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:23.916443    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"]
Jul 10 14:41:23.920367 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:23.920335    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:23.926085 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:23.926055    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-feed7a-serving-cert\""
Jul 10 14:41:23.937456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:23.937428    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"]
Jul 10 14:41:23.963008 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:23.962972    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:24.012595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.012552    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87ac7964-45ab-4897-93b4-af3ab841b43b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.012785 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.012617    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnlrv\" (UniqueName: \"kubernetes.io/projected/87ac7964-45ab-4897-93b4-af3ab841b43b-kube-api-access-fnlrv\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.012785 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.012710    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.064997 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.064964    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:24.114196 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.114104    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnlrv\" (UniqueName: \"kubernetes.io/projected/87ac7964-45ab-4897-93b4-af3ab841b43b-kube-api-access-fnlrv\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.114196 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.114177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.114397 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.114243    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87ac7964-45ab-4897-93b4-af3ab841b43b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.114397 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:41:24.114323    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-serving-cert: secret "maas-api-e2e-shared-b-feed7a-serving-cert" not found
Jul 10 14:41:24.114397 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:41:24.114386    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls podName:87ac7964-45ab-4897-93b4-af3ab841b43b nodeName:}" failed. No retries permitted until 2026-07-10 14:41:24.61437096 +0000 UTC m=+1956.240096868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls") pod "maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" (UID: "87ac7964-45ab-4897-93b4-af3ab841b43b") : secret "maas-api-e2e-shared-b-feed7a-serving-cert" not found
Jul 10 14:41:24.114926 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.114905    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87ac7964-45ab-4897-93b4-af3ab841b43b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.125161 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.125133    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnlrv\" (UniqueName: \"kubernetes.io/projected/87ac7964-45ab-4897-93b4-af3ab841b43b-kube-api-access-fnlrv\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.619456 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.619418    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:24.619666 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:41:24.619535    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-serving-cert: secret "maas-api-e2e-shared-b-feed7a-serving-cert" not found
Jul 10 14:41:24.619666 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:41:24.619591    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls podName:87ac7964-45ab-4897-93b4-af3ab841b43b nodeName:}" failed. No retries permitted until 2026-07-10 14:41:25.619575489 +0000 UTC m=+1957.245301400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls") pod "maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" (UID: "87ac7964-45ab-4897-93b4-af3ab841b43b") : secret "maas-api-e2e-shared-b-feed7a-serving-cert" not found
Jul 10 14:41:24.745991 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.745939    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:24.895045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:24.894946    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:25.637287 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:25.637244    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:25.639637 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:25.639615    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:25.732069 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:25.732029    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:25.870154 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:25.870122    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"]
Jul 10 14:41:25.872254 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:41:25.872227    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ac7964_45ab_4897_93b4_af3ab841b43b.slice/crio-54d82a93e6f62239099cf38646a9a6a9d4cf811083b1bca4838da6599d018b13 WatchSource:0}: Error finding container 54d82a93e6f62239099cf38646a9a6a9d4cf811083b1bca4838da6599d018b13: Status 404 returned error can't find the container with id 54d82a93e6f62239099cf38646a9a6a9d4cf811083b1bca4838da6599d018b13
Jul 10 14:41:26.805767 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:26.805709    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" event={"ID":"87ac7964-45ab-4897-93b4-af3ab841b43b","Type":"ContainerStarted","Data":"757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8"}
Jul 10 14:41:26.806165 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:26.805782    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" event={"ID":"87ac7964-45ab-4897-93b4-af3ab841b43b","Type":"ContainerStarted","Data":"54d82a93e6f62239099cf38646a9a6a9d4cf811083b1bca4838da6599d018b13"}
Jul 10 14:41:26.806165 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:26.805803    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:26.827493 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:26.827430    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" podStartSLOduration=3.410864775 podStartE2EDuration="3.827406928s" totalImagesPullingTime="416.542153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:41:26.825685587 +0000 UTC m=+1958.451411545" watchObservedRunningTime="2026-07-10 14:41:26.827406928 +0000 UTC m=+1958.453132860"
Jul 10 14:41:28.340929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:28.340897    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:32.815044 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:32.815015    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:41:38.102435 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:38.102399    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:38.200007 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:38.199975    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:40.837232 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:40.837154    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:45.660174 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:45.660129    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:45.760875 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:45.760838    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:47.586320 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.586287    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"]
Jul 10 14:41:47.589868 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.589850    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.595275 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.595255    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"kube-root-ca.crt\""
Jul 10 14:41:47.596580 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.596561    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"auth-test-model-9e5d79-kserve-self-signed-certs\""
Jul 10 14:41:47.596738 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.596632    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"default-dockercfg-85sdv\""
Jul 10 14:41:47.598776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.598758    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"openshift-service-ca.crt\""
Jul 10 14:41:47.632094 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.632065    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"]
Jul 10 14:41:47.644143 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.644118    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-home\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.644266 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.644159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-tmp-dir\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.644319 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.644268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4wjb\" (UniqueName: \"kubernetes.io/projected/3f58d104-9fd4-425f-b4a4-a63909827e9b-kube-api-access-w4wjb\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.644389 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.644365    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f58d104-9fd4-425f-b4a4-a63909827e9b-tls-certs\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.644442 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.644426    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-dshm\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.644485 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.644453    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-model-cache\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.651426 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.651404    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:41:47.745862 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.745832    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4wjb\" (UniqueName: \"kubernetes.io/projected/3f58d104-9fd4-425f-b4a4-a63909827e9b-kube-api-access-w4wjb\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746032 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.745874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f58d104-9fd4-425f-b4a4-a63909827e9b-tls-certs\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746032 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.745905    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-dshm\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746153 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.746024    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-model-cache\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746153 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.746142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-home\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746261 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.746178    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-tmp-dir\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.746512    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-model-cache\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746673 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.746545    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-home\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.746673 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.746565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-tmp-dir\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.748513 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.748478    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f58d104-9fd4-425f-b4a4-a63909827e9b-tls-certs\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.748634 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.748519    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-dshm\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.756028 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.756005    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4wjb\" (UniqueName: \"kubernetes.io/projected/3f58d104-9fd4-425f-b4a4-a63909827e9b-kube-api-access-w4wjb\") pod \"auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") " pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:47.901239 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:47.901156    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:48.039157 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:48.039130    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"]
Jul 10 14:41:48.041859 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:41:48.041832    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f58d104_9fd4_425f_b4a4_a63909827e9b.slice/crio-98c2b67b8ec80fda7718a9ed57e7f58279e9f958cf638e992db65ca98abd0777 WatchSource:0}: Error finding container 98c2b67b8ec80fda7718a9ed57e7f58279e9f958cf638e992db65ca98abd0777: Status 404 returned error can't find the container with id 98c2b67b8ec80fda7718a9ed57e7f58279e9f958cf638e992db65ca98abd0777
Jul 10 14:41:48.908532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:48.908496    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5" event={"ID":"3f58d104-9fd4-425f-b4a4-a63909827e9b","Type":"ContainerStarted","Data":"9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6"}
Jul 10 14:41:48.908532 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:48.908538    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5" event={"ID":"3f58d104-9fd4-425f-b4a4-a63909827e9b","Type":"ContainerStarted","Data":"98c2b67b8ec80fda7718a9ed57e7f58279e9f958cf638e992db65ca98abd0777"}
Jul 10 14:41:48.927649 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:48.927605    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5" podStartSLOduration=1.927589356 podStartE2EDuration="1.927589356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:41:48.926008841 +0000 UTC m=+1980.551734763" watchObservedRunningTime="2026-07-10 14:41:48.927589356 +0000 UTC m=+1980.553315287"
Jul 10 14:41:57.901570 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:57.901539    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:57.901992 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:57.901581    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:57.914347 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:57.914320    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:41:57.960885 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:41:57.960857    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:42:26.412264 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:42:26.412234    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:37.051519 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:42:37.051485    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:37.557365 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:42:37.557330    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:42:48.452168 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:42:48.452129    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:43:48.920668 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:43:48.920637    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:43:48.924321 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:43:48.924298    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hz6t_597072ba-b1e2-4d2e-baa8-a253fefb6284/ovn-acl-logging/0.log"
Jul 10 14:44:23.623341 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:23.623297    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:24.789732 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:24.789697    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"]
Jul 10 14:44:24.804623 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:24.804596    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:25.770083 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:25.770051    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:30.699314 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:30.699284    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:44:39.809165 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:39.809108    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5" podUID="3f58d104-9fd4-425f-b4a4-a63909827e9b" containerName="main" containerID="cri-o://9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6" gracePeriod=45
Jul 10 14:44:40.057033 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.057011    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:44:40.168608 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168538    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-model-cache\") pod \"3f58d104-9fd4-425f-b4a4-a63909827e9b\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") "
Jul 10 14:44:40.168608 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168591    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4wjb\" (UniqueName: \"kubernetes.io/projected/3f58d104-9fd4-425f-b4a4-a63909827e9b-kube-api-access-w4wjb\") pod \"3f58d104-9fd4-425f-b4a4-a63909827e9b\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") "
Jul 10 14:44:40.168832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168623    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-home\") pod \"3f58d104-9fd4-425f-b4a4-a63909827e9b\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") "
Jul 10 14:44:40.168832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168681    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-tmp-dir\") pod \"3f58d104-9fd4-425f-b4a4-a63909827e9b\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") "
Jul 10 14:44:40.168832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168706    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f58d104-9fd4-425f-b4a4-a63909827e9b-tls-certs\") pod \"3f58d104-9fd4-425f-b4a4-a63909827e9b\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") "
Jul 10 14:44:40.168832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168806    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f58d104-9fd4-425f-b4a4-a63909827e9b" (UID: "3f58d104-9fd4-425f-b4a4-a63909827e9b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:44:40.168832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168820    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-dshm\") pod \"3f58d104-9fd4-425f-b4a4-a63909827e9b\" (UID: \"3f58d104-9fd4-425f-b4a4-a63909827e9b\") "
Jul 10 14:44:40.169064 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168947    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-home" (OuterVolumeSpecName: "home") pod "3f58d104-9fd4-425f-b4a4-a63909827e9b" (UID: "3f58d104-9fd4-425f-b4a4-a63909827e9b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:44:40.169064 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.168984    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f58d104-9fd4-425f-b4a4-a63909827e9b" (UID: "3f58d104-9fd4-425f-b4a4-a63909827e9b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:44:40.169173 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.169116    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-model-cache\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:44:40.169173 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.169128    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-home\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:44:40.169173 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.169137    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-tmp-dir\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:44:40.171011 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.170967    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-dshm" (OuterVolumeSpecName: "dshm") pod "3f58d104-9fd4-425f-b4a4-a63909827e9b" (UID: "3f58d104-9fd4-425f-b4a4-a63909827e9b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:44:40.171129 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.171036    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f58d104-9fd4-425f-b4a4-a63909827e9b-kube-api-access-w4wjb" (OuterVolumeSpecName: "kube-api-access-w4wjb") pod "3f58d104-9fd4-425f-b4a4-a63909827e9b" (UID: "3f58d104-9fd4-425f-b4a4-a63909827e9b"). InnerVolumeSpecName "kube-api-access-w4wjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:44:40.171129 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.171043    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f58d104-9fd4-425f-b4a4-a63909827e9b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f58d104-9fd4-425f-b4a4-a63909827e9b" (UID: "3f58d104-9fd4-425f-b4a4-a63909827e9b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:44:40.269696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.269666    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f58d104-9fd4-425f-b4a4-a63909827e9b-dshm\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:44:40.269696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.269692    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4wjb\" (UniqueName: \"kubernetes.io/projected/3f58d104-9fd4-425f-b4a4-a63909827e9b-kube-api-access-w4wjb\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:44:40.269696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.269703    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f58d104-9fd4-425f-b4a4-a63909827e9b-tls-certs\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:44:40.695983 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.695947    2584 generic.go:356] "Generic (PLEG): container finished" podID="3f58d104-9fd4-425f-b4a4-a63909827e9b" containerID="9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6" exitCode=0
Jul 10 14:44:40.696167 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.695992    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"
Jul 10 14:44:40.696167 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.696035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5" event={"ID":"3f58d104-9fd4-425f-b4a4-a63909827e9b","Type":"ContainerDied","Data":"9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6"}
Jul 10 14:44:40.696167 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.696086    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5" event={"ID":"3f58d104-9fd4-425f-b4a4-a63909827e9b","Type":"ContainerDied","Data":"98c2b67b8ec80fda7718a9ed57e7f58279e9f958cf638e992db65ca98abd0777"}
Jul 10 14:44:40.696167 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.696104    2584 scope.go:117] "RemoveContainer" containerID="9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6"
Jul 10 14:44:40.705344 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.705312    2584 scope.go:117] "RemoveContainer" containerID="9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6"
Jul 10 14:44:40.705586 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:44:40.705569    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6\": container with ID starting with 9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6 not found: ID does not exist" containerID="9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6"
Jul 10 14:44:40.705634 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.705596    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6"} err="failed to get container status \"9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6\": rpc error: code = NotFound desc = could not find container \"9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6\": container with ID starting with 9882fc196df960538749cbadbcb903f9fd122b8d978d5cfaa952b92311a310f6 not found: ID does not exist"
Jul 10 14:44:40.720645 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.720623    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"]
Jul 10 14:44:40.725799 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.725780    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/auth-test-model-9e5d79-kserve-79584bdb5f-6qvt5"]
Jul 10 14:44:40.921663 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:44:40.921624    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f58d104-9fd4-425f-b4a4-a63909827e9b" path="/var/lib/kubelet/pods/3f58d104-9fd4-425f-b4a4-a63909827e9b/volumes"
Jul 10 14:45:12.171068 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.171035    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"]
Jul 10 14:45:12.171496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.171370    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f58d104-9fd4-425f-b4a4-a63909827e9b" containerName="main"
Jul 10 14:45:12.171496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.171380    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f58d104-9fd4-425f-b4a4-a63909827e9b" containerName="main"
Jul 10 14:45:12.171496 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.171434    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f58d104-9fd4-425f-b4a4-a63909827e9b" containerName="main"
Jul 10 14:45:12.174466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.174449    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.178055 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.178032    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-feed7a\"/\"openshift-service-ca.crt\""
Jul 10 14:45:12.178271 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.178255    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-feed7a\"/\"default-dockercfg-rq5bd\""
Jul 10 14:45:12.179391 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.179377    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-feed7a\"/\"kube-root-ca.crt\""
Jul 10 14:45:12.179569 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.179513    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-feed7a\"/\"sub-test-model-feed7a-kserve-self-signed-certs\""
Jul 10 14:45:12.185595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.185577    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"]
Jul 10 14:45:12.201823 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.201805    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:12.233940 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.233916    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tmp-dir\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.234028 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.233951    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-model-cache\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.234028 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.233974    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tls-certs\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.234117 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.234028    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-dshm\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.234117 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.234058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-home\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.234186 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.234116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgq29\" (UniqueName: \"kubernetes.io/projected/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-kube-api-access-vgq29\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335341 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-model-cache\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335481 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335352    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tls-certs\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335481 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335398    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-dshm\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335481 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335432    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-home\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335481 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335464    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgq29\" (UniqueName: \"kubernetes.io/projected/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-kube-api-access-vgq29\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335495    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tmp-dir\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335789 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335712    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-model-cache\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335847 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335813    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-home\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.335906 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.335889    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tmp-dir\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.337616 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.337590    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-dshm\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.337906 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.337890    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tls-certs\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.343704 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.343681    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgq29\" (UniqueName: \"kubernetes.io/projected/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-kube-api-access-vgq29\") pod \"sub-test-model-feed7a-kserve-66787f5fdc-s2kc2\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") " pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.483164 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.483089    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:12.612389 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.612368    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"]
Jul 10 14:45:12.614018 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:45:12.613990    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad22cab9_26d7_443f_ab81_c4e63fbd7fd9.slice/crio-b57c9cd9f74faba527d66e1f30d7b90d6a48540ee5869a7770dcc47975754360 WatchSource:0}: Error finding container b57c9cd9f74faba527d66e1f30d7b90d6a48540ee5869a7770dcc47975754360: Status 404 returned error can't find the container with id b57c9cd9f74faba527d66e1f30d7b90d6a48540ee5869a7770dcc47975754360
Jul 10 14:45:12.840802 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.840770    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2" event={"ID":"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9","Type":"ContainerStarted","Data":"cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8"}
Jul 10 14:45:12.840973 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.840808    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2" event={"ID":"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9","Type":"ContainerStarted","Data":"b57c9cd9f74faba527d66e1f30d7b90d6a48540ee5869a7770dcc47975754360"}
Jul 10 14:45:12.861047 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:12.861001    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2" podStartSLOduration=0.860986312 podStartE2EDuration="860.986312ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:12.85960803 +0000 UTC m=+2184.485333962" watchObservedRunningTime="2026-07-10 14:45:12.860986312 +0000 UTC m=+2184.486712241"
Jul 10 14:45:22.483240 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:22.483202    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:22.483240 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:22.483243    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:22.495614 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:22.495594    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:22.898447 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:22.898418    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:45:59.205324 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:59.205287    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:45:59.305296 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:45:59.305261    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:00.279807 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:00.279772    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:01.052070 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:01.052038    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"]
Jul 10 14:46:01.071159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:01.071123    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:16.072144 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.072048    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2" podUID="ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" containerName="main" containerID="cri-o://cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8" gracePeriod=45
Jul 10 14:46:16.327003 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.326951    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:46:16.507065 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507031    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-dshm\") pod \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") "
Jul 10 14:46:16.507249 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507124    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-model-cache\") pod \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") "
Jul 10 14:46:16.507249 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507162    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgq29\" (UniqueName: \"kubernetes.io/projected/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-kube-api-access-vgq29\") pod \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") "
Jul 10 14:46:16.507249 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507211    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-home\") pod \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") "
Jul 10 14:46:16.507249 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507239    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tls-certs\") pod \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") "
Jul 10 14:46:16.507534 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507337    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tmp-dir\") pod \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\" (UID: \"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9\") "
Jul 10 14:46:16.507534 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507382    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" (UID: "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:46:16.507534 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507459    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-home" (OuterVolumeSpecName: "home") pod "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" (UID: "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:46:16.507689 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507589    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-model-cache\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:46:16.507689 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507608    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-home\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:46:16.507689 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.507619    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" (UID: "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:46:16.509359 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.509327    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-dshm" (OuterVolumeSpecName: "dshm") pod "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" (UID: "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:46:16.509486 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.509464    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" (UID: "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:46:16.509577 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.509556    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-kube-api-access-vgq29" (OuterVolumeSpecName: "kube-api-access-vgq29") pod "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" (UID: "ad22cab9-26d7-443f-ab81-c4e63fbd7fd9"). InnerVolumeSpecName "kube-api-access-vgq29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:46:16.609058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.608978    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgq29\" (UniqueName: \"kubernetes.io/projected/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-kube-api-access-vgq29\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:46:16.609058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.609004    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tls-certs\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:46:16.609058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.609015    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-tmp-dir\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:46:16.609058 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:16.609023    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9-dshm\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:46:17.139390 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.139355    2584 generic.go:356] "Generic (PLEG): container finished" podID="ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" containerID="cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8" exitCode=0
Jul 10 14:46:17.139832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.139398    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"
Jul 10 14:46:17.139832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.139449    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2" event={"ID":"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9","Type":"ContainerDied","Data":"cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8"}
Jul 10 14:46:17.139832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.139490    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2" event={"ID":"ad22cab9-26d7-443f-ab81-c4e63fbd7fd9","Type":"ContainerDied","Data":"b57c9cd9f74faba527d66e1f30d7b90d6a48540ee5869a7770dcc47975754360"}
Jul 10 14:46:17.139832 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.139508    2584 scope.go:117] "RemoveContainer" containerID="cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8"
Jul 10 14:46:17.147732 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.147701    2584 scope.go:117] "RemoveContainer" containerID="cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8"
Jul 10 14:46:17.147995 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:46:17.147977    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8\": container with ID starting with cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8 not found: ID does not exist" containerID="cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8"
Jul 10 14:46:17.148066 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.148001    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8"} err="failed to get container status \"cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8\": rpc error: code = NotFound desc = could not find container \"cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8\": container with ID starting with cdf557332ca37c5530e04c40252930d092ba5431144271d4ab07e6cc67dc4bc8 not found: ID does not exist"
Jul 10 14:46:17.159713 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.159691    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"]
Jul 10 14:46:17.168807 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:17.168786    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-feed7a/sub-test-model-feed7a-kserve-66787f5fdc-s2kc2"]
Jul 10 14:46:18.921695 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:18.921661    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" path="/var/lib/kubelet/pods/ad22cab9-26d7-443f-ab81-c4e63fbd7fd9/volumes"
Jul 10 14:46:21.577672 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.577638    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"]
Jul 10 14:46:21.578131 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.577993    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" containerName="main"
Jul 10 14:46:21.578131 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.578004    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" containerName="main"
Jul 10 14:46:21.578131 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.578076    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad22cab9-26d7-443f-ab81-c4e63fbd7fd9" containerName="main"
Jul 10 14:46:21.582217 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.582202    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.586355 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.586330    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"default-dockercfg-85sdv\""
Jul 10 14:46:21.586355 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.586342    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"rate-test-model-9e5d79-kserve-self-signed-certs\""
Jul 10 14:46:21.586565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.586368    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"openshift-service-ca.crt\""
Jul 10 14:46:21.586565 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.586454    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9e5d79\"/\"kube-root-ca.crt\""
Jul 10 14:46:21.590020 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.589999    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"]
Jul 10 14:46:21.649894 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.649866    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-dshm\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.650027 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.649924    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq6kd\" (UniqueName: \"kubernetes.io/projected/6662b1a3-f2b1-4131-bee2-ee34b966642b-kube-api-access-gq6kd\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.650027 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.649985    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-model-cache\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.650027 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.650007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-tmp-dir\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.650148 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.650104    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-home\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.650189 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.650150    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6662b1a3-f2b1-4131-bee2-ee34b966642b-tls-certs\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.686106 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.686080    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:21.751280 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751252    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq6kd\" (UniqueName: \"kubernetes.io/projected/6662b1a3-f2b1-4131-bee2-ee34b966642b-kube-api-access-gq6kd\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751392 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751291    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-model-cache\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751392 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-tmp-dir\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751392 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751365    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-home\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751530 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751399    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6662b1a3-f2b1-4131-bee2-ee34b966642b-tls-certs\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751579 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751546    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-dshm\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751758 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751710    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-model-cache\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751756    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-tmp-dir\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.751886 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.751799    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-home\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.753568 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.753550    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-dshm\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.753872 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.753855    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6662b1a3-f2b1-4131-bee2-ee34b966642b-tls-certs\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.760587 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.760561    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq6kd\" (UniqueName: \"kubernetes.io/projected/6662b1a3-f2b1-4131-bee2-ee34b966642b-kube-api-access-gq6kd\") pod \"rate-test-model-9e5d79-kserve-785585dc79-sdrgq\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") " pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:21.892528 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:21.892464    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:22.026296 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:22.026263    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"]
Jul 10 14:46:22.029470 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:46:22.029443    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6662b1a3_f2b1_4131_bee2_ee34b966642b.slice/crio-2d48b619a185542ae77fc42fe1e8e9651abd93ce7707a391d85f691e01426e42 WatchSource:0}: Error finding container 2d48b619a185542ae77fc42fe1e8e9651abd93ce7707a391d85f691e01426e42: Status 404 returned error can't find the container with id 2d48b619a185542ae77fc42fe1e8e9651abd93ce7707a391d85f691e01426e42
Jul 10 14:46:22.165618 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:22.165525    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq" event={"ID":"6662b1a3-f2b1-4131-bee2-ee34b966642b","Type":"ContainerStarted","Data":"3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7"}
Jul 10 14:46:22.165618 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:22.165569    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq" event={"ID":"6662b1a3-f2b1-4131-bee2-ee34b966642b","Type":"ContainerStarted","Data":"2d48b619a185542ae77fc42fe1e8e9651abd93ce7707a391d85f691e01426e42"}
Jul 10 14:46:22.187688 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:22.187643    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq" podStartSLOduration=1.187628088 podStartE2EDuration="1.187628088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:22.186238471 +0000 UTC m=+2253.811964401" watchObservedRunningTime="2026-07-10 14:46:22.187628088 +0000 UTC m=+2253.813354056"
Jul 10 14:46:31.892627 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:31.892588    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:31.893255 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:31.892644    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:31.905162 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:31.905130    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:32.225490 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:32.225398    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:46:32.659994 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:32.659960    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:32.731663 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:32.731628    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:37.721494 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:37.721464    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:39.069112 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:39.069071    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:39.234816 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:39.234779    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:46.535191 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:46.535154    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:46:46.638225 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:46:46.638190    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:27.088166 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:27.088128    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"]
Jul 10 14:47:27.113366 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:27.113311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:27.975929 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:27.975894    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:42.107985 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.107878    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq" podUID="6662b1a3-f2b1-4131-bee2-ee34b966642b" containerName="main" containerID="cri-o://3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7" gracePeriod=45
Jul 10 14:47:42.213194 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.213145    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq" podUID="6662b1a3-f2b1-4131-bee2-ee34b966642b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": dial tcp 10.134.0.53:8000: connect: connection refused"
Jul 10 14:47:42.359569 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.359505    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:47:42.479747 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.479677    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-tmp-dir\") pod \"6662b1a3-f2b1-4131-bee2-ee34b966642b\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") "
Jul 10 14:47:42.479943 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.479769    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-home\") pod \"6662b1a3-f2b1-4131-bee2-ee34b966642b\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") "
Jul 10 14:47:42.479943 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.479804    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-dshm\") pod \"6662b1a3-f2b1-4131-bee2-ee34b966642b\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") "
Jul 10 14:47:42.479943 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.479846    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-model-cache\") pod \"6662b1a3-f2b1-4131-bee2-ee34b966642b\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") "
Jul 10 14:47:42.479943 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.479872    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6662b1a3-f2b1-4131-bee2-ee34b966642b-tls-certs\") pod \"6662b1a3-f2b1-4131-bee2-ee34b966642b\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") "
Jul 10 14:47:42.479943 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.479934    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gq6kd\" (UniqueName: \"kubernetes.io/projected/6662b1a3-f2b1-4131-bee2-ee34b966642b-kube-api-access-gq6kd\") pod \"6662b1a3-f2b1-4131-bee2-ee34b966642b\" (UID: \"6662b1a3-f2b1-4131-bee2-ee34b966642b\") "
Jul 10 14:47:42.480183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.479951    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6662b1a3-f2b1-4131-bee2-ee34b966642b" (UID: "6662b1a3-f2b1-4131-bee2-ee34b966642b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:47:42.480183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.480045    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-home" (OuterVolumeSpecName: "home") pod "6662b1a3-f2b1-4131-bee2-ee34b966642b" (UID: "6662b1a3-f2b1-4131-bee2-ee34b966642b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:47:42.480183 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.480091    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6662b1a3-f2b1-4131-bee2-ee34b966642b" (UID: "6662b1a3-f2b1-4131-bee2-ee34b966642b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:47:42.480300 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.480218    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-tmp-dir\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:42.480300 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.480231    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-home\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:42.480300 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.480240    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-model-cache\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:42.481969 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.481944    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6662b1a3-f2b1-4131-bee2-ee34b966642b-kube-api-access-gq6kd" (OuterVolumeSpecName: "kube-api-access-gq6kd") pod "6662b1a3-f2b1-4131-bee2-ee34b966642b" (UID: "6662b1a3-f2b1-4131-bee2-ee34b966642b"). InnerVolumeSpecName "kube-api-access-gq6kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:47:42.482070 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.481987    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-dshm" (OuterVolumeSpecName: "dshm") pod "6662b1a3-f2b1-4131-bee2-ee34b966642b" (UID: "6662b1a3-f2b1-4131-bee2-ee34b966642b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:47:42.482070 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.481991    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6662b1a3-f2b1-4131-bee2-ee34b966642b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6662b1a3-f2b1-4131-bee2-ee34b966642b" (UID: "6662b1a3-f2b1-4131-bee2-ee34b966642b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:47:42.541764 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.541729    2584 generic.go:356] "Generic (PLEG): container finished" podID="6662b1a3-f2b1-4131-bee2-ee34b966642b" containerID="3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7" exitCode=0
Jul 10 14:47:42.541927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.541783    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"
Jul 10 14:47:42.541927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.541825    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq" event={"ID":"6662b1a3-f2b1-4131-bee2-ee34b966642b","Type":"ContainerDied","Data":"3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7"}
Jul 10 14:47:42.541927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.541873    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq" event={"ID":"6662b1a3-f2b1-4131-bee2-ee34b966642b","Type":"ContainerDied","Data":"2d48b619a185542ae77fc42fe1e8e9651abd93ce7707a391d85f691e01426e42"}
Jul 10 14:47:42.541927 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.541890    2584 scope.go:117] "RemoveContainer" containerID="3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7"
Jul 10 14:47:42.551133 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.551115    2584 scope.go:117] "RemoveContainer" containerID="3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7"
Jul 10 14:47:42.551423 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:47:42.551405    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7\": container with ID starting with 3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7 not found: ID does not exist" containerID="3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7"
Jul 10 14:47:42.551472 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.551432    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7"} err="failed to get container status \"3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7\": rpc error: code = NotFound desc = could not find container \"3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7\": container with ID starting with 3c1235499a66da298a6785b6f17a95f04628449a841f6b07068804cb77d549f7 not found: ID does not exist"
Jul 10 14:47:42.565415 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.565378    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"]
Jul 10 14:47:42.569790 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.569760    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9e5d79/rate-test-model-9e5d79-kserve-785585dc79-sdrgq"]
Jul 10 14:47:42.581037 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.581010    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6662b1a3-f2b1-4131-bee2-ee34b966642b-dshm\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:42.581171 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.581060    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6662b1a3-f2b1-4131-bee2-ee34b966642b-tls-certs\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:42.581171 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.581075    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq6kd\" (UniqueName: \"kubernetes.io/projected/6662b1a3-f2b1-4131-bee2-ee34b966642b-kube-api-access-gq6kd\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:42.922594 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:42.922561    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6662b1a3-f2b1-4131-bee2-ee34b966642b" path="/var/lib/kubelet/pods/6662b1a3-f2b1-4131-bee2-ee34b966642b/volumes"
Jul 10 14:47:47.482548 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:47.482512    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:47.964923 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:47.964879    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:48.096776 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.096742    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:48.197691 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.197658    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:48.378773 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.378742    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"]
Jul 10 14:47:48.379030 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.379000    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56" podUID="346ab15b-a527-4c88-a46a-4c3bca94b4df" containerName="maas-api" containerID="cri-o://e0346e24afccec555c23d4b5367692079f7d338a3b26ec750d99ac43eb39685f" gracePeriod=30
Jul 10 14:47:48.401045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.401021    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:48.571905 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.571870    2584 generic.go:356] "Generic (PLEG): container finished" podID="346ab15b-a527-4c88-a46a-4c3bca94b4df" containerID="e0346e24afccec555c23d4b5367692079f7d338a3b26ec750d99ac43eb39685f" exitCode=0
Jul 10 14:47:48.572250 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.571944    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56" event={"ID":"346ab15b-a527-4c88-a46a-4c3bca94b4df","Type":"ContainerDied","Data":"e0346e24afccec555c23d4b5367692079f7d338a3b26ec750d99ac43eb39685f"}
Jul 10 14:47:48.627132 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.627108    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:47:48.734670 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.734603    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/346ab15b-a527-4c88-a46a-4c3bca94b4df-maas-api-tls\") pod \"346ab15b-a527-4c88-a46a-4c3bca94b4df\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") "
Jul 10 14:47:48.734860 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.734682    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/346ab15b-a527-4c88-a46a-4c3bca94b4df-openshift-service-ca\") pod \"346ab15b-a527-4c88-a46a-4c3bca94b4df\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") "
Jul 10 14:47:48.734860 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.734743    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsghg\" (UniqueName: \"kubernetes.io/projected/346ab15b-a527-4c88-a46a-4c3bca94b4df-kube-api-access-wsghg\") pod \"346ab15b-a527-4c88-a46a-4c3bca94b4df\" (UID: \"346ab15b-a527-4c88-a46a-4c3bca94b4df\") "
Jul 10 14:47:48.735067 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.735046    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346ab15b-a527-4c88-a46a-4c3bca94b4df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "346ab15b-a527-4c88-a46a-4c3bca94b4df" (UID: "346ab15b-a527-4c88-a46a-4c3bca94b4df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:47:48.736625 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.736602    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346ab15b-a527-4c88-a46a-4c3bca94b4df-kube-api-access-wsghg" (OuterVolumeSpecName: "kube-api-access-wsghg") pod "346ab15b-a527-4c88-a46a-4c3bca94b4df" (UID: "346ab15b-a527-4c88-a46a-4c3bca94b4df"). InnerVolumeSpecName "kube-api-access-wsghg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:47:48.736696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.736602    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/346ab15b-a527-4c88-a46a-4c3bca94b4df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "346ab15b-a527-4c88-a46a-4c3bca94b4df" (UID: "346ab15b-a527-4c88-a46a-4c3bca94b4df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:47:48.836121 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.836091    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/346ab15b-a527-4c88-a46a-4c3bca94b4df-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:48.836121 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.836119    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/346ab15b-a527-4c88-a46a-4c3bca94b4df-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:48.836121 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:48.836130    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsghg\" (UniqueName: \"kubernetes.io/projected/346ab15b-a527-4c88-a46a-4c3bca94b4df-kube-api-access-wsghg\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:49.065120 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:49.065099    2584 scope.go:117] "RemoveContainer" containerID="e0346e24afccec555c23d4b5367692079f7d338a3b26ec750d99ac43eb39685f"
Jul 10 14:47:49.576595 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:49.576567    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"
Jul 10 14:47:49.576974 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:49.576565    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56" event={"ID":"346ab15b-a527-4c88-a46a-4c3bca94b4df","Type":"ContainerDied","Data":"ec0621e29f9f0f762e5ad8536f3d7415b2deddb8421f5da00400d03183e9bd12"}
Jul 10 14:47:49.609459 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:49.609430    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"]
Jul 10 14:47:49.613730 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:49.613694    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9e5d79-78877b65bb-r2h56"]
Jul 10 14:47:50.921514 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:50.921479    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346ab15b-a527-4c88-a46a-4c3bca94b4df" path="/var/lib/kubelet/pods/346ab15b-a527-4c88-a46a-4c3bca94b4df/volumes"
Jul 10 14:47:54.105387 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:54.105355    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:54.207759 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:54.207700    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:54.428022 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:54.427940    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:54.686525 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:54.686450    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:54.951966 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:54.951883    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:55.332951 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:55.332919    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:55.524466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:55.524434    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:55.629209 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:55.629126    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:55.817038 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:55.817008    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"]
Jul 10 14:47:55.817233 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:55.817189    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" podUID="87ac7964-45ab-4897-93b4-af3ab841b43b" containerName="maas-api" containerID="cri-o://757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8" gracePeriod=30
Jul 10 14:47:55.855907 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:55.855882    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:47:56.066323 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.066303    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:47:56.208036 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.207950    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnlrv\" (UniqueName: \"kubernetes.io/projected/87ac7964-45ab-4897-93b4-af3ab841b43b-kube-api-access-fnlrv\") pod \"87ac7964-45ab-4897-93b4-af3ab841b43b\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") "
Jul 10 14:47:56.208036 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.208012    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls\") pod \"87ac7964-45ab-4897-93b4-af3ab841b43b\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") "
Jul 10 14:47:56.208036 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.208032    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87ac7964-45ab-4897-93b4-af3ab841b43b-openshift-service-ca\") pod \"87ac7964-45ab-4897-93b4-af3ab841b43b\" (UID: \"87ac7964-45ab-4897-93b4-af3ab841b43b\") "
Jul 10 14:47:56.208457 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.208428    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87ac7964-45ab-4897-93b4-af3ab841b43b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87ac7964-45ab-4897-93b4-af3ab841b43b" (UID: "87ac7964-45ab-4897-93b4-af3ab841b43b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:47:56.210019 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.209997    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87ac7964-45ab-4897-93b4-af3ab841b43b-kube-api-access-fnlrv" (OuterVolumeSpecName: "kube-api-access-fnlrv") pod "87ac7964-45ab-4897-93b4-af3ab841b43b" (UID: "87ac7964-45ab-4897-93b4-af3ab841b43b"). InnerVolumeSpecName "kube-api-access-fnlrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:47:56.210137 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.210121    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87ac7964-45ab-4897-93b4-af3ab841b43b" (UID: "87ac7964-45ab-4897-93b4-af3ab841b43b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:47:56.308815 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.308768    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnlrv\" (UniqueName: \"kubernetes.io/projected/87ac7964-45ab-4897-93b4-af3ab841b43b-kube-api-access-fnlrv\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:56.308815 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.308807    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87ac7964-45ab-4897-93b4-af3ab841b43b-maas-api-tls\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:56.308815 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.308820    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87ac7964-45ab-4897-93b4-af3ab841b43b-openshift-service-ca\") on node \"ip-10-0-143-123.ec2.internal\" DevicePath \"\""
Jul 10 14:47:56.610504 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.610471    2584 generic.go:356] "Generic (PLEG): container finished" podID="87ac7964-45ab-4897-93b4-af3ab841b43b" containerID="757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8" exitCode=0
Jul 10 14:47:56.611001 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.610516    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"
Jul 10 14:47:56.611001 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.610543    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" event={"ID":"87ac7964-45ab-4897-93b4-af3ab841b43b","Type":"ContainerDied","Data":"757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8"}
Jul 10 14:47:56.611001 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.610583    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g" event={"ID":"87ac7964-45ab-4897-93b4-af3ab841b43b","Type":"ContainerDied","Data":"54d82a93e6f62239099cf38646a9a6a9d4cf811083b1bca4838da6599d018b13"}
Jul 10 14:47:56.611001 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.610604    2584 scope.go:117] "RemoveContainer" containerID="757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8"
Jul 10 14:47:56.619184 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.619166    2584 scope.go:117] "RemoveContainer" containerID="757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8"
Jul 10 14:47:56.619459 ip-10-0-143-123 kubenswrapper[2584]: E0710 14:47:56.619436    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8\": container with ID starting with 757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8 not found: ID does not exist" containerID="757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8"
Jul 10 14:47:56.619518 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.619471    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8"} err="failed to get container status \"757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8\": rpc error: code = NotFound desc = could not find container \"757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8\": container with ID starting with 757652c2e96e206c7d00d970f064d29036ecc874fee0a8a93698df550dc75ed8 not found: ID does not exist"
Jul 10 14:47:56.635751 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.635710    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"]
Jul 10 14:47:56.641344 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.641322    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feed7a-6d4fc54c98-bxt6g"]
Jul 10 14:47:56.920439 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:47:56.920359    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87ac7964-45ab-4897-93b4-af3ab841b43b" path="/var/lib/kubelet/pods/87ac7964-45ab-4897-93b4-af3ab841b43b/volumes"
Jul 10 14:48:01.591572 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:01.591536    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:01.786171 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:01.786138    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:01.940673 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:01.940589    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:02.113335 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:02.113302    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:02.526875 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:02.526834    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dcbb7dc9c-ph7jd_267e171c-f86e-482d-806c-736b0ab32fd7/authorino/0.log"
Jul 10 14:48:08.386323 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:08.386292    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d47b746d6-dwhl4_d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8/copy-manifests/0.log"
Jul 10 14:48:08.396588 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:08.396565    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d47b746d6-dwhl4_d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8/manager/0.log"
Jul 10 14:48:08.880549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:08.880502    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68fc5bb9d9-6xdsf_a03d95c8-6cb7-4b14-8045-52afa8679b5d/maas-api/0.log"
Jul 10 14:48:10.375540 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:10.375508    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dcbb7dc9c-ph7jd_267e171c-f86e-482d-806c-736b0ab32fd7/authorino/0.log"
Jul 10 14:48:10.761683 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:10.761602    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hns75_b4f36db8-fb3a-46a5-a0d6-98d5abae8562/kuadrant-console-plugin/0.log"
Jul 10 14:48:11.011310 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:11.011280    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-p8pxd_4f35817c-d7bb-48dc-a478-a2cdd8bf2d66/manager/0.log"
Jul 10 14:48:11.936304 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:11.936276    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545bb46c8-9gsz5_8bbe80d9-d6a2-4f74-921e-76173b3ae289/kube-auth-proxy/0.log"
Jul 10 14:48:12.456376 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:12.456344    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cch5m_8f9190d8-cc06-4879-a490-56a43de21be8/payload-processing/0.log"
Jul 10 14:48:13.177692 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:13.177648    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rvk86_25e6d4b8-11ae-464a-9523-5ad58c45694d/main/0.log"
Jul 10 14:48:13.542783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:13.542703    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-vqkqg_39d7dfaf-3c69-4e0e-a244-e0ba3789296b/main/0.log"
Jul 10 14:48:17.636904 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:17.636866    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:17.727982 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:17.727949    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log" limit=1024
Jul 10 14:48:20.558208 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:20.558178    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cm2np_259a5382-782c-4cba-8bee-effd8d3b49fc/global-pull-secret-syncer/0.log"
Jul 10 14:48:20.668774 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:20.668738    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5j7xb_cae3993c-e09a-4c84-8d16-6eed8dff45dd/konnectivity-agent/0.log"
Jul 10 14:48:20.819914 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:20.819887    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-123.ec2.internal_7490a82eb930c707bbdadeb01bf134d0/haproxy/0.log"
Jul 10 14:48:26.234019 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:26.233986    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dcbb7dc9c-ph7jd_267e171c-f86e-482d-806c-736b0ab32fd7/authorino/0.log"
Jul 10 14:48:26.335184 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:26.335158    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hns75_b4f36db8-fb3a-46a5-a0d6-98d5abae8562/kuadrant-console-plugin/0.log"
Jul 10 14:48:26.532194 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:26.532118    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-p8pxd_4f35817c-d7bb-48dc-a478-a2cdd8bf2d66/manager/0.log"
Jul 10 14:48:28.703804 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:28.703774    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d4fc76bcd-5m4gs_5b565e78-cded-4cd1-9283-30cf82be7997/metrics-server/0.log"
Jul 10 14:48:28.724597 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:28.724574    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-qxpqs_f057f08c-6ce1-448e-a548-0af7467f0739/monitoring-plugin/0.log"
Jul 10 14:48:28.918466 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:28.918441    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8tpr_f3f41fdc-56bd-4be1-915a-6c2a15f84169/node-exporter/0.log"
Jul 10 14:48:28.940285 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:28.940261    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8tpr_f3f41fdc-56bd-4be1-915a-6c2a15f84169/kube-rbac-proxy/0.log"
Jul 10 14:48:28.965045 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:28.964987    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m8tpr_f3f41fdc-56bd-4be1-915a-6c2a15f84169/init-textfile/0.log"
Jul 10 14:48:29.081769 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.081741    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8998f81-303c-420a-88e3-a5e7031d074f/prometheus/0.log"
Jul 10 14:48:29.102803 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.102781    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8998f81-303c-420a-88e3-a5e7031d074f/config-reloader/0.log"
Jul 10 14:48:29.127953 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.127914    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8998f81-303c-420a-88e3-a5e7031d074f/thanos-sidecar/0.log"
Jul 10 14:48:29.149292 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.149271    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8998f81-303c-420a-88e3-a5e7031d074f/kube-rbac-proxy-web/0.log"
Jul 10 14:48:29.186316 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.186298    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8998f81-303c-420a-88e3-a5e7031d074f/kube-rbac-proxy/0.log"
Jul 10 14:48:29.209267 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.209246    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8998f81-303c-420a-88e3-a5e7031d074f/kube-rbac-proxy-thanos/0.log"
Jul 10 14:48:29.231528 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.231466    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8998f81-303c-420a-88e3-a5e7031d074f/init-config-reloader/0.log"
Jul 10 14:48:29.338109 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.338083    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78b8b5d5bd-x8fql_7f7669a7-7bfe-4698-b6af-120ce0f7e7d1/telemeter-client/0.log"
Jul 10 14:48:29.359739 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.359695    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78b8b5d5bd-x8fql_7f7669a7-7bfe-4698-b6af-120ce0f7e7d1/reload/0.log"
Jul 10 14:48:29.382668 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.382634    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78b8b5d5bd-x8fql_7f7669a7-7bfe-4698-b6af-120ce0f7e7d1/kube-rbac-proxy/0.log"
Jul 10 14:48:29.417323 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.417299    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6845b84675-k9755_adad8394-cc75-4339-9898-a25835beef4f/thanos-query/0.log"
Jul 10 14:48:29.438209 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.438184    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6845b84675-k9755_adad8394-cc75-4339-9898-a25835beef4f/kube-rbac-proxy-web/0.log"
Jul 10 14:48:29.459214 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.459193    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6845b84675-k9755_adad8394-cc75-4339-9898-a25835beef4f/kube-rbac-proxy/0.log"
Jul 10 14:48:29.480141 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.480110    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6845b84675-k9755_adad8394-cc75-4339-9898-a25835beef4f/prom-label-proxy/0.log"
Jul 10 14:48:29.505358 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.505289    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6845b84675-k9755_adad8394-cc75-4339-9898-a25835beef4f/kube-rbac-proxy-rules/0.log"
Jul 10 14:48:29.529441 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:29.529420    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6845b84675-k9755_adad8394-cc75-4339-9898-a25835beef4f/kube-rbac-proxy-metrics/0.log"
Jul 10 14:48:30.508390 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508360    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"]
Jul 10 14:48:30.508783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508709    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ac7964-45ab-4897-93b4-af3ab841b43b" containerName="maas-api"
Jul 10 14:48:30.508783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508734    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ac7964-45ab-4897-93b4-af3ab841b43b" containerName="maas-api"
Jul 10 14:48:30.508783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508766    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346ab15b-a527-4c88-a46a-4c3bca94b4df" containerName="maas-api"
Jul 10 14:48:30.508783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508775    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="346ab15b-a527-4c88-a46a-4c3bca94b4df" containerName="maas-api"
Jul 10 14:48:30.508783 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508783    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6662b1a3-f2b1-4131-bee2-ee34b966642b" containerName="main"
Jul 10 14:48:30.508949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508792    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6662b1a3-f2b1-4131-bee2-ee34b966642b" containerName="main"
Jul 10 14:48:30.508949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508848    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="87ac7964-45ab-4897-93b4-af3ab841b43b" containerName="maas-api"
Jul 10 14:48:30.508949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508856    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="346ab15b-a527-4c88-a46a-4c3bca94b4df" containerName="maas-api"
Jul 10 14:48:30.508949 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.508864    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6662b1a3-f2b1-4131-bee2-ee34b966642b" containerName="main"
Jul 10 14:48:30.511444 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.511421    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.514633 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.514609    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hrtpc\"/\"default-dockercfg-sf26v\""
Jul 10 14:48:30.515875 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.515856    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hrtpc\"/\"kube-root-ca.crt\""
Jul 10 14:48:30.515972 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.515885    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hrtpc\"/\"openshift-service-ca.crt\""
Jul 10 14:48:30.520095 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.520068    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"]
Jul 10 14:48:30.607050 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.607019    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-lib-modules\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.607187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.607058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbcbn\" (UniqueName: \"kubernetes.io/projected/857c6c57-67a8-49ae-96ce-5b1aaf159301-kube-api-access-cbcbn\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.607187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.607086    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-podres\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.607187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.607109    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-sys\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.607187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.607170    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-proc\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.707976 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.707943    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-proc\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.707997    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-lib-modules\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.708036    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbcbn\" (UniqueName: \"kubernetes.io/projected/857c6c57-67a8-49ae-96ce-5b1aaf159301-kube-api-access-cbcbn\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.708077    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-podres\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.708081    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-proc\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.708102    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-sys\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708159 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.708154    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-sys\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708395 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.708189    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-lib-modules\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.708395 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.708196    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/857c6c57-67a8-49ae-96ce-5b1aaf159301-podres\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.717115 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.717095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbcbn\" (UniqueName: \"kubernetes.io/projected/857c6c57-67a8-49ae-96ce-5b1aaf159301-kube-api-access-cbcbn\") pod \"perf-node-gather-daemonset-6d5kx\" (UID: \"857c6c57-67a8-49ae-96ce-5b1aaf159301\") " pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.822588 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.822568    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:30.949019 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:30.948997    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"]
Jul 10 14:48:30.950859 ip-10-0-143-123 kubenswrapper[2584]: W0710 14:48:30.950823    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod857c6c57_67a8_49ae_96ce_5b1aaf159301.slice/crio-d60c7e77579fbb3d117f665497d6e275f9dac6f1a8e4352179f759037daadb3f WatchSource:0}: Error finding container d60c7e77579fbb3d117f665497d6e275f9dac6f1a8e4352179f759037daadb3f: Status 404 returned error can't find the container with id d60c7e77579fbb3d117f665497d6e275f9dac6f1a8e4352179f759037daadb3f
Jul 10 14:48:31.769562 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:31.769526    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx" event={"ID":"857c6c57-67a8-49ae-96ce-5b1aaf159301","Type":"ContainerStarted","Data":"a3f3c5cf73e0ff6e1b320a8275a4a419407af84c7bbb3d4d7226cab1bd723498"}
Jul 10 14:48:31.770059 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:31.769581    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx" event={"ID":"857c6c57-67a8-49ae-96ce-5b1aaf159301","Type":"ContainerStarted","Data":"d60c7e77579fbb3d117f665497d6e275f9dac6f1a8e4352179f759037daadb3f"}
Jul 10 14:48:31.770059 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:31.769636    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:31.786549 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:31.786503    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx" podStartSLOduration=1.786488822 podStartE2EDuration="1.786488822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:48:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:48:31.78598382 +0000 UTC m=+2383.411709752" watchObservedRunningTime="2026-07-10 14:48:31.786488822 +0000 UTC m=+2383.412214752"
Jul 10 14:48:33.624187 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:33.624161    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hcxqk_f45056ea-8aa6-4200-a036-2591f6364b9a/dns/0.log"
Jul 10 14:48:33.654346 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:33.654316    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hcxqk_f45056ea-8aa6-4200-a036-2591f6364b9a/kube-rbac-proxy/0.log"
Jul 10 14:48:33.791696 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:33.791672    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t2kz9_b72c716d-8ba2-4fdf-b1dc-ecfcbb0e1e5d/dns-node-resolver/0.log"
Jul 10 14:48:34.518414 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:34.518384    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qjk28_d918dd0c-1b5a-49be-b8cf-fe2049a2efac/node-ca/0.log"
Jul 10 14:48:35.916087 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:35.916057    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545bb46c8-9gsz5_8bbe80d9-d6a2-4f74-921e-76173b3ae289/kube-auth-proxy/0.log"
Jul 10 14:48:36.123286 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:36.123256    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cch5m_8f9190d8-cc06-4879-a490-56a43de21be8/payload-processing/0.log"
Jul 10 14:48:36.906455 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:36.906427    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kvltq_cd39c349-df40-4932-92b7-a29e8c11e195/serve-healthcheck-canary/0.log"
Jul 10 14:48:37.556790 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:37.556760    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjb47_75c35095-9063-4aa7-a7c8-4e518a91821b/kube-rbac-proxy/0.log"
Jul 10 14:48:37.585581 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:37.585551    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjb47_75c35095-9063-4aa7-a7c8-4e518a91821b/exporter/0.log"
Jul 10 14:48:37.615626 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:37.615600    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fjb47_75c35095-9063-4aa7-a7c8-4e518a91821b/extractor/0.log"
Jul 10 14:48:37.783797 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:37.783768    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hrtpc/perf-node-gather-daemonset-6d5kx"
Jul 10 14:48:40.961852 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:40.961804    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d47b746d6-dwhl4_d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8/manager/0.log"
Jul 10 14:48:40.992662 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:40.992600    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d47b746d6-dwhl4_d2104d9c-1677-4ac0-9ca8-ffec4d93cfd8/copy-manifests/0.log"
Jul 10 14:48:41.013277 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:41.013250    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545bb46c8-9gsz5_8bbe80d9-d6a2-4f74-921e-76173b3ae289/kube-auth-proxy/0.log"
Jul 10 14:48:41.157996 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:41.157971    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cch5m_8f9190d8-cc06-4879-a490-56a43de21be8/payload-processing/0.log"
Jul 10 14:48:42.363476 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:42.363449    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5d5554d48-98lhq_28ed8289-8054-4cc4-ac14-8e19410a6dbf/manager/0.log"
Jul 10 14:48:42.391130 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:42.391100    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hl92h_3a119259-101e-4f7a-b95f-7a0b022c9fe7/openshift-lws-operator/0.log"
Jul 10 14:48:42.596169 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:42.596146    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545bb46c8-9gsz5_8bbe80d9-d6a2-4f74-921e-76173b3ae289/kube-auth-proxy/0.log"
Jul 10 14:48:42.747302 ip-10-0-143-123 kubenswrapper[2584]: I0710 14:48:42.747273    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-cch5m_8f9190d8-cc06-4879-a490-56a43de21be8/payload-processing/0.log"