Jul 02 19:54:45.358863 ip-10-0-142-101 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 02 19:54:45.358875 ip-10-0-142-101 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 02 19:54:45.358881 ip-10-0-142-101 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 02 19:54:45.359101 ip-10-0-142-101 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 02 19:54:55.387865 ip-10-0-142-101 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 02 19:54:55.387879 ip-10-0-142-101 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 157606d490c94c709ccc9887a87e066c --
Jul 02 19:56:27.724559 ip-10-0-142-101 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 19:56:28.167741 ip-10-0-142-101 kubenswrapper[2536]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:56:28.167741 ip-10-0-142-101 kubenswrapper[2536]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 19:56:28.167741 ip-10-0-142-101 kubenswrapper[2536]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:56:28.167741 ip-10-0-142-101 kubenswrapper[2536]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 19:56:28.167741 ip-10-0-142-101 kubenswrapper[2536]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 19:56:28.168911 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.168830    2536 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 19:56:28.176316 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176299    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:56:28.176316 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176316    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176320    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176324    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176327    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176330    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176335    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176338    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176341    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176344    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176346    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176348    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176351    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176354    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176356    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176359    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176361    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176364    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176366    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176369    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176371    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:56:28.176385 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176373    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176378    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176390    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176393    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176396    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176399    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176402    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176405    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176408    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176411    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176414    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176416    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176419    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176421    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176424    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176426    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176428    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176431    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176433    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176436    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:56:28.176850 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176439    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176441    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176443    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176446    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176448    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176450    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176453    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176455    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176457    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176460    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176462    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176464    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176467    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176469    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176472    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176475    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176478    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176480    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176482    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176485    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:56:28.177345 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176487    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176490    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176494    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176496    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176499    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176501    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176504    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176506    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176508    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176511    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176514    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176516    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176519    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176541    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176545    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176549    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176551    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176554    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176557    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176559    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:56:28.177820 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176562    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176565    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176569    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176572    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176574    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176580    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176582    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176585    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176588    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176590    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176594    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176597    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176600    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176602    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176604    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176607    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176609    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.176613    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177300    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:56:28.178322 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177306    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177308    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177311    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177314    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177319    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177323    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177326    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177329    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177332    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177335    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177338    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177341    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177343    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177346    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177349    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177353    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177356    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177359    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177362    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:56:28.178819 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177365    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177368    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177370    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177373    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177376    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177378    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177381    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177383    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177386    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177389    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177391    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177393    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177396    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177398    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177400    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177403    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177406    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177409    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177412    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177414    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:56:28.179297 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177417    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177419    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177422    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177424    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177427    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177429    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177432    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177434    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177437    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177439    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177442    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177445    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177448    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177451    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177453    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177455    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177459    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177461    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177464    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:56:28.179786 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177466    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177470    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177474    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177477    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177480    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177483    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177486    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177488    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177491    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177498    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177500    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177503    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177505    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177508    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177510    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177513    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177515    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177518    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177521    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177523    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:56:28.180247 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177526    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177529    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177531    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177533    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177537    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177540    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177542    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177545    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177547    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177549    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177553    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177555    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177557    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177561    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177564    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177567    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177569    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177571    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177574    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177576    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:56:28.180733 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177936    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177940    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177943    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177946    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177949    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177952    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177954    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177956    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177959    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177962    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177967    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177969    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177972    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177975    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177977    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177979    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177982    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177985    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177987    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:56:28.181214 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177990    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177993    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177995    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.177998    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178001    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178003    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178005    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178008    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178010    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178013    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178015    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178018    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178020    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178022    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178025    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178027    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178029    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178032    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178034    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178037    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:56:28.181683 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178039    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178043    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178046    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178050    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178053    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178056    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178059    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178063    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178065    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178068    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178070    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178073    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178075    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178077    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178080    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178082    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178086    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178088    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178091    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:56:28.182163 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178093    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178095    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178098    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178100    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178103    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178105    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178107    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178110    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178112    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178116    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178119    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178122    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178126    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178128    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178131    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178134    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178139    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178141    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178144    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178146    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:56:28.182635 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178149    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178151    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178154    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178156    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178159    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178161    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178163    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178166    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178168    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178171    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178174    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178176    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178178    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178181    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178183    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178186    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178188    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178192    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178195    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:56:28.183146 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178198    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178201    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178296    2536 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178307    2536 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178315    2536 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178319    2536 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178322    2536 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178325    2536 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178330    2536 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178343    2536 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178347    2536 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178350    2536 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178353    2536 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178356    2536 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178359    2536 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178362    2536 flags.go:64] FLAG: --cgroup-root=""
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178365    2536 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178368    2536 flags.go:64] FLAG: --client-ca-file=""
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178371    2536 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178374    2536 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178378    2536 flags.go:64] FLAG: --cluster-domain=""
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178381    2536 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178384    2536 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178386    2536 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 19:56:28.183608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178390    2536 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178394    2536 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178398    2536 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178400    2536 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178404    2536 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178406    2536 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178409    2536 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178412    2536 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178415    2536 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178418    2536 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178422    2536 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178424    2536 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178427    2536 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178430    2536 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178432    2536 flags.go:64] FLAG: --enable-server="true"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178437    2536 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178441    2536 flags.go:64] FLAG: --event-burst="100"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178446    2536 flags.go:64] FLAG: --event-qps="50"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178449    2536 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178451    2536 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178454    2536 flags.go:64] FLAG: --eviction-hard=""
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178458    2536 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178461    2536 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178464    2536 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178467    2536 flags.go:64] FLAG: --eviction-soft=""
Jul 02 19:56:28.184179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178469    2536 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178472    2536 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178474    2536 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178477    2536 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178480    2536 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178483    2536 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178485    2536 flags.go:64] FLAG: --feature-gates=""
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178489    2536 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178492    2536 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178495    2536 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178498    2536 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178501    2536 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178504    2536 flags.go:64] FLAG: --help="false"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178506    2536 flags.go:64] FLAG: --hostname-override="ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178509    2536 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178512    2536 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178515    2536 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178518    2536 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178521    2536 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178523    2536 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178526    2536 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178528    2536 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178532    2536 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 19:56:28.184784 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178535    2536 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178540    2536 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178542    2536 flags.go:64] FLAG: --kube-reserved=""
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178545    2536 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178548    2536 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178551    2536 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178553    2536 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178556    2536 flags.go:64] FLAG: --lock-file=""
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178559    2536 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178562    2536 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178565    2536 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178569    2536 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178572    2536 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178575    2536 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178578    2536 flags.go:64] FLAG: --logging-format="text"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178580    2536 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178583    2536 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178586    2536 flags.go:64] FLAG: --manifest-url=""
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178589    2536 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178594    2536 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178597    2536 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178601    2536 flags.go:64] FLAG: --max-pods="110"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178604    2536 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178606    2536 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178609    2536 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 19:56:28.185463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178612    2536 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178615    2536 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178618    2536 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178620    2536 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178627    2536 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178630    2536 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178634    2536 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178637    2536 flags.go:64] FLAG: --pod-cidr=""
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178641    2536 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178647    2536 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178649    2536 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178653    2536 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178655    2536 flags.go:64] FLAG: --port="10250"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178658    2536 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178661    2536 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03ac00386e412165e"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178664    2536 flags.go:64] FLAG: --qos-reserved=""
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178667    2536 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178669    2536 flags.go:64] FLAG: --register-node="true"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178672    2536 flags.go:64] FLAG: --register-with-taints=""
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178676    2536 flags.go:64] FLAG: --registry-burst="10"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178678    2536 flags.go:64] FLAG: --registry-qps="5"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178681    2536 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178684    2536 flags.go:64] FLAG: --reserved-memory=""
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178687    2536 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178690    2536 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 19:56:28.186372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178693    2536 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178696    2536 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178699    2536 flags.go:64] FLAG: --runonce="false"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178702    2536 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178704    2536 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178707    2536 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178710    2536 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178712    2536 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178715    2536 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178718    2536 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178721    2536 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178723    2536 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178726    2536 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178730    2536 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178733    2536 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178736    2536 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178740    2536 flags.go:64] FLAG: --system-cgroups=""
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178743    2536 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178748    2536 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178750    2536 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178753    2536 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178756    2536 flags.go:64] FLAG: --tls-min-version=""
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178759    2536 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178762    2536 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178764    2536 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 19:56:28.187342 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178767    2536 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178770    2536 flags.go:64] FLAG: --v="2"
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178773    2536 flags.go:64] FLAG: --version="false"
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178777    2536 flags.go:64] FLAG: --vmodule=""
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178781    2536 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.178784    2536 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178898    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178903    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178905    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178909    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178911    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178914    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178916    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178919    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178922    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178924    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178926    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178929    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178931    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178934    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178938    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:56:28.187944 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178940    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178943    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178946    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178948    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178950    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178953    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178955    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178957    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178960    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178962    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178965    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178967    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178969    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178972    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178974    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178976    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178980    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178984    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178986    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:56:28.188466 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178989    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178992    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178994    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178997    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.178999    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179001    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179004    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179006    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179009    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179011    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179013    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179015    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179019    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179023    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179027    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179030    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179032    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179035    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179037    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179040    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:56:28.188913 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179042    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179044    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179047    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179049    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179051    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179054    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179056    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179058    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179061    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179064    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179068    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179070    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179074    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179076    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179079    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179082    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179085    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179087    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179090    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:56:28.189408 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179092    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179094    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179097    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179100    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179102    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179105    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179108    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179110    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179113    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179115    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179117    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179120    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179122    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179124    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179127    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179130    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179132    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179135    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179137    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179139    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:56:28.189863 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179142    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179144    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179146    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179149    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179151    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.179154    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.179902    2536 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.188597    2536 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.188613    2536 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.188637    2536 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.188644    2536 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188700    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188705    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188709    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188712    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:56:28.190370 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188715    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188718    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188722    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188724    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188727    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188730    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188732    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188735    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188737    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188740    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188743    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188745    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188748    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188750    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188753    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188755    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188758    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188761    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188765    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:56:28.190745 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188768    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188770    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188773    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188775    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188778    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188780    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188784    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188787    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188789    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188792    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188794    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188797    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188799    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188801    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188804    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188806    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188808    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188811    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188814    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188816    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:56:28.191219 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188819    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188821    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188823    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188826    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188829    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188831    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188834    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188836    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188838    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188841    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188843    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188845    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188848    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188850    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188852    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188855    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188857    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188859    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188863    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188865    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:56:28.191778 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188867    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188870    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188872    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188875    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188879    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188882    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188885    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188887    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188890    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188892    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188895    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188898    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188901    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188903    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188906    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188908    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188911    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188915    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188918    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:56:28.192350 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188921    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188924    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188927    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188929    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188932    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188934    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188937    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188939    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188942    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188944    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188946    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188949    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188952    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188954    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188956    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188959    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:56:28.192803 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.188961    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.188967    2536 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189081    2536 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189085    2536 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189088    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189091    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189094    2536 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189096    2536 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189099    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189101    2536 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189104    2536 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189106    2536 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189109    2536 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189111    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189115    2536 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 19:56:28.193223 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189117    2536 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189120    2536 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189122    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189124    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189127    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189129    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189132    2536 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189134    2536 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189138    2536 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189141    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189144    2536 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189147    2536 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189150    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189153    2536 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189156    2536 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189159    2536 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189161    2536 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189164    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189166    2536 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 19:56:28.193621 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189169    2536 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189171    2536 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189174    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189176    2536 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189179    2536 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189181    2536 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189183    2536 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189186    2536 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189189    2536 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189191    2536 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189193    2536 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189196    2536 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189198    2536 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189202    2536 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189204    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189207    2536 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189209    2536 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189211    2536 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189214    2536 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 19:56:28.194085 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189216    2536 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189219    2536 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189222    2536 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189224    2536 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189226    2536 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189229    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189233    2536 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189236    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189239    2536 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189242    2536 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189244    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189246    2536 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189249    2536 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189251    2536 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189254    2536 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189256    2536 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189258    2536 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189261    2536 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189263    2536 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189265    2536 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 19:56:28.194555 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189268    2536 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189270    2536 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189273    2536 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189275    2536 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189277    2536 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189280    2536 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189303    2536 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189310    2536 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189313    2536 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189315    2536 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189318    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189320    2536 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189322    2536 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189325    2536 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189328    2536 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189330    2536 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189333    2536 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189335    2536 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189338    2536 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 19:56:28.195039 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189342    2536 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189345    2536 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189349    2536 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189351    2536 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189354    2536 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189356    2536 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189359    2536 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189361    2536 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.189363    2536 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.189368    2536 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.189465    2536 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.191849    2536 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.193404    2536 server.go:1019] "Starting client certificate rotation"
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.193500    2536 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 19:56:28.195633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.193531    2536 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 19:56:28.196515 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.196501    2536 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 19:56:28.198329 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.198304    2536 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 19:56:28.209535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.209518    2536 log.go:25] "Validated CRI v1 runtime API"
Jul 02 19:56:28.212243 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.212215    2536 log.go:25] "Validated CRI v1 image API"
Jul 02 19:56:28.217679 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.217666    2536 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 19:56:28.220177 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.220159    2536 fs.go:135] Filesystem UUIDs: map[018f6ecc-0aa2-4f9b-95f7-b1687a507b77:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 deeb3c53-2958-4d62-9ca1-40eca0e7d06d:/dev/nvme0n1p3]
Jul 02 19:56:28.220240 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.220177    2536 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 19:56:28.222842 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.222827    2536 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 19:56:28.227001 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.226898    2536 manager.go:216] Machine: {Timestamp:2026-07-02 19:56:28.225475103 +0000 UTC m=+0.373243873 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099949 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec250b1634497fefe5318de7f51ca56d SystemUUID:ec250b16-3449-7fef-e531-8de7f51ca56d BootID:157606d4-90c9-4c70-9ccc-9887a87e066c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:54:27:de:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:54:27:de:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:5e:37:14:ad:4a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 19:56:28.227001 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.226995    2536 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 19:56:28.227141 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.227068    2536 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 19:56:28.230483 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.230455    2536 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 19:56:28.230621 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.230485    2536 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 19:56:28.232052 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.232042    2536 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 19:56:28.232096 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.232054    2536 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 19:56:28.232096 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.232062    2536 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 19:56:28.232936 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.232925    2536 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 19:56:28.234901 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.234891    2536 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 19:56:28.239367 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239356    2536 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 19:56:28.239477 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239468    2536 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 19:56:28.239507 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239499    2536 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 19:56:28.239541 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239509    2536 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 19:56:28.239541 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239520    2536 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 19:56:28.239541 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239528    2536 kubelet.go:398] "Adding apiserver pod source"
Jul 02 19:56:28.239541 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239537    2536 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 19:56:28.239902 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239891    2536 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 19:56:28.239934 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.239909    2536 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 19:56:28.242257 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.242238    2536 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 19:56:28.244539 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.244525    2536 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 19:56:28.244613 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.244556    2536 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 19:56:28.245957 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245940    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 19:56:28.245957 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245958    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245964    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245970    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245979    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245984    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245992    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.245997    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.246003    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.246009    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.246016    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.246022    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.246055    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 19:56:28.246098 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.246060    2536 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 19:56:28.249402 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.249389    2536 server.go:1268] "Started kubelet"
Jul 02 19:56:28.249554 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.249503    2536 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 19:56:28.249650 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.249564    2536 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 19:56:28.249650 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.249628    2536 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 19:56:28.249825 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.249803    2536 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 19:56:28.249886 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.249807    2536 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 19:56:28.249989 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.249967    2536 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 19:56:28.250567 ip-10-0-142-101 systemd[1]: Started Kubernetes Kubelet.
Jul 02 19:56:28.251814 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.251793    2536 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 19:56:28.252007 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.251994    2536 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 19:56:28.255472 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.255449    2536 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 19:56:28.256176 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.255039    2536 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-101.ec2.internal.18be913bbd3def1c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-101.ec2.internal,UID:ip-10-0-142-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-101.ec2.internal,},FirstTimestamp:2026-07-02 19:56:28.24934582 +0000 UTC m=+0.397114590,LastTimestamp:2026-07-02 19:56:28.24934582 +0000 UTC m=+0.397114590,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-101.ec2.internal,}"
Jul 02 19:56:28.260153 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.260139    2536 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 19:56:28.260209 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.260154    2536 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 19:56:28.260266 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.260215    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.260998 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.260947    2536 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 19:56:28.260998 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.260967    2536 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 19:56:28.260998 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.260948    2536 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 19:56:28.260998 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261037    2536 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 19:56:28.260998 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261046    2536 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 19:56:28.261608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261593    2536 factory.go:55] Registering systemd factory
Jul 02 19:56:28.261608 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261610    2536 factory.go:223] Registration of the systemd container factory successfully
Jul 02 19:56:28.261802 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261790    2536 factory.go:153] Registering CRI-O factory
Jul 02 19:56:28.261802 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261801    2536 factory.go:223] Registration of the crio container factory successfully
Jul 02 19:56:28.261869 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261841    2536 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 19:56:28.261869 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261858    2536 factory.go:103] Registering Raw factory
Jul 02 19:56:28.261869 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.261867    2536 manager.go:1196] Started watching for new ooms in manager
Jul 02 19:56:28.262481 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.262468    2536 manager.go:349] Starting recovery of all containers
Jul 02 19:56:28.262712 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.262685    2536 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 02 19:56:28.262805 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.262765    2536 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 02 19:56:28.263702 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.263646    2536 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-49nrd"
Jul 02 19:56:28.272247 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.272047    2536 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-49nrd"
Jul 02 19:56:28.273120 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:28.273107    2536 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.max": read /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service/memory.max: no such device
Jul 02 19:56:28.273521 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.273507    2536 manager.go:354] Recovery completed
Jul 02 19:56:28.278094 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.278082    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:56:28.279891 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.279877    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:56:28.279952 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.279902    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:56:28.279952 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.279912    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:56:28.280495 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.280479    2536 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 19:56:28.280569 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.280499    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:56:28.280569 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.280498    2536 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 19:56:28.280569 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.280556    2536 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 19:56:28.281124 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.281110    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:56:28.281176 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.281134    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:56:28.281176 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.281144    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:56:28.282657 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.282641    2536 policy_none.go:49] "None policy: Start"
Jul 02 19:56:28.282711 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.282665    2536 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 19:56:28.282711 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.282681    2536 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 19:56:28.284831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.284819    2536 policy_none.go:47] "Start"
Jul 02 19:56:28.324862 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.324843    2536 manager.go:337] "Starting Device Plugin manager"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.324884    2536 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.324894    2536 server.go:87] "Starting device plugin registration server"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.325161    2536 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.325173    2536 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.325321    2536 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.325397    2536 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.325406    2536 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.326016    2536 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 19:56:28.345212 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.326068    2536 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.361222 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.361195    2536 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 19:56:28.362462 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.362444    2536 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 19:56:28.362529 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.362482    2536 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 19:56:28.362529 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.362502    2536 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 19:56:28.362606 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.362530    2536 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 02 19:56:28.364989 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.364972    2536 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:56:28.425987 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.425919    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:56:28.426987 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.426971    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:56:28.427074 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.427000    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:56:28.427074 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.427010    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:56:28.427074 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.427033    2536 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.435057 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.435036    2536 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.435151 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.435060    2536 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-101.ec2.internal\": node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.462827 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.462800    2536 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal"]
Jul 02 19:56:28.462993 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.462881    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:56:28.462993 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.462904    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.464329 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.464314    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:56:28.464416 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.464346    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:56:28.464416 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.464357    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:56:28.466708 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.466695    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:56:28.466862 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.466847    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.466925 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.466883    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:56:28.467357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.467338    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:56:28.467435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.467367    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:56:28.467435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.467344    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:56:28.467435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.467379    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:56:28.467435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.467396    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:56:28.467435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.467408    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:56:28.469627 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.469613    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.469681 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.469637    2536 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 19:56:28.470262 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.470246    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 19:56:28.470346 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.470277    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 19:56:28.470346 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.470307    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeHasSufficientPID"
Jul 02 19:56:28.494878 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.494857    2536 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-101.ec2.internal\" not found" node="ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.499224 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.499211    2536 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-101.ec2.internal\" not found" node="ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.562220 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.562199    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de540d23a4bee0040959efe51be6b698-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal\" (UID: \"de540d23a4bee0040959efe51be6b698\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.562344 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.562229    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de540d23a4bee0040959efe51be6b698-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal\" (UID: \"de540d23a4bee0040959efe51be6b698\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.562344 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.562247    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a63f3713d25ef95e553ae4a5cda922f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-101.ec2.internal\" (UID: \"6a63f3713d25ef95e553ae4a5cda922f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.563235 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.563219    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.662796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.662757    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de540d23a4bee0040959efe51be6b698-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal\" (UID: \"de540d23a4bee0040959efe51be6b698\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.662796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.662797    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de540d23a4bee0040959efe51be6b698-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal\" (UID: \"de540d23a4bee0040959efe51be6b698\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.662953 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.662818    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a63f3713d25ef95e553ae4a5cda922f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-101.ec2.internal\" (UID: \"6a63f3713d25ef95e553ae4a5cda922f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.662953 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.662865    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de540d23a4bee0040959efe51be6b698-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal\" (UID: \"de540d23a4bee0040959efe51be6b698\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.662953 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.662864    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6a63f3713d25ef95e553ae4a5cda922f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-101.ec2.internal\" (UID: \"6a63f3713d25ef95e553ae4a5cda922f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.662953 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.662870    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de540d23a4bee0040959efe51be6b698-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal\" (UID: \"de540d23a4bee0040959efe51be6b698\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.663824 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.663801    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.764576 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.764504    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.798690 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.798662    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.802567 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:28.802544    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:28.864796 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.864767    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:28.965268 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:28.965240    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:29.065664 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:29.065614    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:29.166143 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:29.166103    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:29.192212 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.192184    2536 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 19:56:29.192756 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.192368    2536 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 19:56:29.266487 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:29.266459    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:29.274941 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.274906    2536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 19:51:28 +0000 UTC" deadline="2027-12-03 21:25:38.728207781 +0000 UTC"
Jul 02 19:56:29.274941 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.274937    2536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12457h29m9.453274099s"
Jul 02 19:56:29.367557 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:29.367533    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:29.468136 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:29.468096    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:29.481381 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:29.481345    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a63f3713d25ef95e553ae4a5cda922f.slice/crio-07c2b6bc640ed594c0426a10d4bfeadedec94e0d37afd97297e4d9b6c2024700 WatchSource:0}: Error finding container 07c2b6bc640ed594c0426a10d4bfeadedec94e0d37afd97297e4d9b6c2024700: Status 404 returned error can't find the container with id 07c2b6bc640ed594c0426a10d4bfeadedec94e0d37afd97297e4d9b6c2024700
Jul 02 19:56:29.481577 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:29.481558    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde540d23a4bee0040959efe51be6b698.slice/crio-a063f7f357b9acb0d68c321b42660c2b0539529bbb8135ca1a3f96eafcaf11ee WatchSource:0}: Error finding container a063f7f357b9acb0d68c321b42660c2b0539529bbb8135ca1a3f96eafcaf11ee: Status 404 returned error can't find the container with id a063f7f357b9acb0d68c321b42660c2b0539529bbb8135ca1a3f96eafcaf11ee
Jul 02 19:56:29.485663 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.485644    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 19:56:29.561339 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.561310    2536 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:56:29.568629 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:29.568607    2536 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-101.ec2.internal\" not found"
Jul 02 19:56:29.601411 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.601363    2536 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:56:29.652102 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.652077    2536 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:56:29.661359 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.661337    2536 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:29.673357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.673340    2536 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 19:56:29.674262 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.674250    2536 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal"
Jul 02 19:56:29.683884 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:29.683868    2536 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 19:56:30.241170 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.241140    2536 apiserver.go:52] "Watching apiserver"
Jul 02 19:56:30.250793 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.250770    2536 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 19:56:30.251313 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.251271    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-ss72c","kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7","openshift-cluster-node-tuning-operator/tuned-qpf9v","openshift-dns/node-resolver-bt4zv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal","openshift-multus/multus-additional-cni-plugins-587br","openshift-multus/network-metrics-daemon-d2pws","openshift-ovn-kubernetes/ovnkube-node-8dzn4","kube-system/global-pull-secret-syncer-llksn","kube-system/konnectivity-agent-hvksz","openshift-image-registry/node-ca-xnztd","openshift-multus/multus-lsq54","openshift-network-diagnostics/network-check-target-7mf8w"]
Jul 02 19:56:30.253827 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.253809    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.255797 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.255779    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.257809 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.257789    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.259812 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.259795    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.260620 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.260595    2536 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 19:56:30.262159 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.262144    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.264142 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.263929    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:30.264142 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.263983    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:30.264696 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.264442    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 19:56:30.264696 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.264465    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 19:56:30.264696 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.264464    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 19:56:30.264696 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.264475    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 19:56:30.264696 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.264525    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-94lgd\""
Jul 02 19:56:30.266128 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.266113    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.266937 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.266919    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 19:56:30.267766 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.267408    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 19:56:30.267766 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.267612    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 19:56:30.267766 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.267619    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 19:56:30.267766 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.267684    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 19:56:30.267766 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.267691    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h7gq8\""
Jul 02 19:56:30.267766 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.267619    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 19:56:30.268056 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.267930    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 19:56:30.268376 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.268362    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.268883 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.268869    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rcstz\""
Jul 02 19:56:30.268955 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.268908    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fcrzp\""
Jul 02 19:56:30.269010 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.268984    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jvkq9\""
Jul 02 19:56:30.269078 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.268984    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 19:56:30.269131 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.269087    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 19:56:30.269789 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.269774    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 19:56:30.270752 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270734    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cf4a098-9d50-4bf7-af7b-576e55e64798-tmp\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.270823 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270762    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxrpt\" (UniqueName: \"kubernetes.io/projected/b9f3b082-85fb-4e61-84eb-11697e1dbb76-kube-api-access-qxrpt\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.270823 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270786    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-os-release\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.270823 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270803    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysctl-d\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270821    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-multus-certs\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270859    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-etc-kubernetes\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270872    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbmv8\" (UniqueName: \"kubernetes.io/projected/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-kube-api-access-qbmv8\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270894    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-system-cni-dir\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270917    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-cni-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270936    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-cnibin\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270955    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-k8s-cni-cncf-io\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.270975 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.270973    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-modprobe-d\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.271334 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271003    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-cni-binary-copy\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.271334 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271026    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqvb2\" (UniqueName: \"kubernetes.io/projected/112bace8-f3ae-4a77-aa75-a40214e6a48d-kube-api-access-mqvb2\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.271334 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271048    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-sys\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.271334 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271130    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 19:56:30.271334 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271260    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 19:56:30.271334 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271331    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271128    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-cnibin\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271449    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-system-cni-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271481    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-cni-bin\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271514    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-sys-fs\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271525    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zp5gp\""
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271540    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysconfig\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271565    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-kubernetes\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.271591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271589    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-var-lib-kubelet\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271613    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9f3b082-85fb-4e61-84eb-11697e1dbb76-tmp-dir\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271639    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-os-release\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271665    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-cni-binary-copy\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271705    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-cni-multus\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271745    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-socket-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271771    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-etc-selinux\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271820    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrt5h\" (UniqueName: \"kubernetes.io/projected/090239e9-6752-41f7-bb38-677b56132111-kube-api-access-wrt5h\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.271935 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271881    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-systemd\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271937    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-run\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271968    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271993    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.271998    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-kubelet\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272021    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-daemon-config\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272042    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272062    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-registration-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272084    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-device-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272115    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-host\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272146    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272156    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272167    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-socket-dir-parent\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272187    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-netns\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272218    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-conf-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272239    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-tuned\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272271    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h77ws\" (UniqueName: \"kubernetes.io/projected/9cf4a098-9d50-4bf7-af7b-576e55e64798-kube-api-access-h77ws\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.272328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272315    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9f3b082-85fb-4e61-84eb-11697e1dbb76-hosts-file\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.272879 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272337    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.272879 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272380    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-hostroot\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.272879 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272402    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysctl-conf\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.272879 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272422    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-lib-modules\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.272879 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272575    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:30.272879 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.272648    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:30.273036 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.272954    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.273036 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.272997    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:30.275386 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.275367    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.277007 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.276990    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 19:56:30.277256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.277242    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.277539 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.277509    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 19:56:30.277636 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.277614    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7pg77\""
Jul 02 19:56:30.277778 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.277765    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 19:56:30.278043 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.278016    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pfq72\""
Jul 02 19:56:30.278449 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.278434    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 19:56:30.281117 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.281104    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 19:56:30.281229 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.281104    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 19:56:30.281355 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.281341    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rq8fb\""
Jul 02 19:56:30.281455 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.281438    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 19:56:30.284643 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.281959    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 19:56:30.284643 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.283069    2536 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 19:56:30.325950 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.325909    2536 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hvchg"
Jul 02 19:56:30.343432 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.343394    2536 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hvchg"
Jul 02 19:56:30.362429 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.362406    2536 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 19:56:30.366749 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.366711    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal" event={"ID":"de540d23a4bee0040959efe51be6b698","Type":"ContainerStarted","Data":"a063f7f357b9acb0d68c321b42660c2b0539529bbb8135ca1a3f96eafcaf11ee"}
Jul 02 19:56:30.367947 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.367916    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal" event={"ID":"6a63f3713d25ef95e553ae4a5cda922f","Type":"ContainerStarted","Data":"07c2b6bc640ed594c0426a10d4bfeadedec94e0d37afd97297e4d9b6c2024700"}
Jul 02 19:56:30.372848 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.372824    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ntpj\" (UniqueName: \"kubernetes.io/projected/f6881e29-762a-4296-8c61-1b1774a6577a-kube-api-access-8ntpj\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:30.372946 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.372864    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-slash\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.372946 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.372894    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-multus-certs\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.372946 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.372917    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-etc-kubernetes\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.372946 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.372938    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbmv8\" (UniqueName: \"kubernetes.io/projected/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-kube-api-access-qbmv8\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.373126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.372960    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-system-cni-dir\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.373126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.372974    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-multus-certs\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.373126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373008    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-system-cni-dir\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.373126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373018    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-etc-kubernetes\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.373126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373058    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-cni-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.373126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373085    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-cni-binary-copy\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.373126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373107    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-sys\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373132    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovnkube-config\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373158    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-cni-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373154    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-env-overrides\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373203    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-cnibin\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373226    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-cni-bin\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373233    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-cnibin\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373202    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-sys\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373249    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-sys-fs\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373297    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysconfig\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373315    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-sys-fs\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373320    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-kubernetes\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373349    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-cni-bin\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373342    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-var-lib-kubelet\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373382    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovnkube-script-lib\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373417    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-os-release\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373425    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-kubernetes\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.373535 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373440    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-cni-binary-copy\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373392    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysconfig\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373463    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cf4a098-9d50-4bf7-af7b-576e55e64798-tmp\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373487    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373513    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-cni-binary-copy\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373517    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-registration-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373542    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-device-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373580    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-var-lib-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373600    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/313b0821-33f6-430d-b1c7-7b1723f5930c-host\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373622    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373654    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-netns\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373680    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-systemd-units\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373759    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-netns\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373798    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-device-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373815    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-registration-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373851    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373877    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-kubelet\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.374265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373878    2536 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373897    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-systemd\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373911    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-run-ovn-kubernetes\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373931    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-cni-bin\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373976    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.373993    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-cnibin\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374013    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-k8s-cni-cncf-io\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374027    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-modprobe-d\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374042    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-cni-binary-copy\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374049    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-etc-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374073    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e880c852-03c2-4730-ae0e-26df38f98598-agent-certs\") pod \"konnectivity-agent-hvksz\" (UID: \"e880c852-03c2-4730-ae0e-26df38f98598\") " pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374093    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/313b0821-33f6-430d-b1c7-7b1723f5930c-serviceca\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374094    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-var-lib-kubelet\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374151    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374197    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374213    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-cnibin\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374269    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-run-k8s-cni-cncf-io\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375002 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374309    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-os-release\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374362    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqvb2\" (UniqueName: \"kubernetes.io/projected/112bace8-f3ae-4a77-aa75-a40214e6a48d-kube-api-access-mqvb2\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374399    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb62578b-c505-45b0-9606-8f94982ec824-iptables-alerter-script\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374425    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-system-cni-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374427    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-modprobe-d\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374465    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-system-cni-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374467    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9f3b082-85fb-4e61-84eb-11697e1dbb76-tmp-dir\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374494    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-node-log\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374600    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e880c852-03c2-4730-ae0e-26df38f98598-konnectivity-ca\") pod \"konnectivity-agent-hvksz\" (UID: \"e880c852-03c2-4730-ae0e-26df38f98598\") " pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374642    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41653b25-2617-4640-a58f-1853517ff74d-kubelet-config\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374668    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqvxg\" (UniqueName: \"kubernetes.io/projected/313b0821-33f6-430d-b1c7-7b1723f5930c-kube-api-access-mqvxg\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374695    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-cni-multus\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374725    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-socket-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374764    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-etc-selinux\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374830    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrt5h\" (UniqueName: \"kubernetes.io/projected/090239e9-6752-41f7-bb38-677b56132111-kube-api-access-wrt5h\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374854    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-systemd\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374874    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-run\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.375740 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374891    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9f3b082-85fb-4e61-84eb-11697e1dbb76-tmp-dir\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374896    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxrpt\" (UniqueName: \"kubernetes.io/projected/b9f3b082-85fb-4e61-84eb-11697e1dbb76-kube-api-access-qxrpt\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374922    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374960    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374981    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-kubelet\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.374987    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-socket-dir\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375000    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-daemon-config\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375117    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/090239e9-6752-41f7-bb38-677b56132111-etc-selinux\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375156    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-run\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375209    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-cni-multus\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375220    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-host\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375250    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h77ws\" (UniqueName: \"kubernetes.io/projected/9cf4a098-9d50-4bf7-af7b-576e55e64798-kube-api-access-h77ws\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375255    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-systemd\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375326    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-host\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375337    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9f3b082-85fb-4e61-84eb-11697e1dbb76-hosts-file\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375366    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-ovn\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375391    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb62578b-c505-45b0-9606-8f94982ec824-host-slash\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.376357 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375439    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-socket-dir-parent\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375464    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-conf-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375493    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-tuned\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375493    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9f3b082-85fb-4e61-84eb-11697e1dbb76-hosts-file\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375571    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovn-node-metrics-cert\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375590    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-daemon-config\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375595    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375639    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41653b25-2617-4640-a58f-1853517ff74d-dbus\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375665    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375693    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-hostroot\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375766    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysctl-conf\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375799    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-lib-modules\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375830    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-run-netns\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375840    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/112bace8-f3ae-4a77-aa75-a40214e6a48d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375858    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-cni-netd\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375885    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmdp6\" (UniqueName: \"kubernetes.io/projected/b53caea2-cf8f-4fe3-9b37-defacc71a768-kube-api-access-jmdp6\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375898    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-socket-dir-parent\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377049 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375915    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-multus-conf-dir\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375914    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-hostroot\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.375967    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq7zs\" (UniqueName: \"kubernetes.io/projected/fb62578b-c505-45b0-9606-8f94982ec824-kube-api-access-rq7zs\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376001    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-os-release\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376024    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysctl-d\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376024    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-host-var-lib-kubelet\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376049    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-log-socket\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376083    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/112bace8-f3ae-4a77-aa75-a40214e6a48d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376093    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376165    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysctl-d\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376196    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-sysctl-conf\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376199    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-os-release\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.377831 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.376216    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cf4a098-9d50-4bf7-af7b-576e55e64798-lib-modules\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.378802 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.378779    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cf4a098-9d50-4bf7-af7b-576e55e64798-etc-tuned\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.378879 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.378808    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cf4a098-9d50-4bf7-af7b-576e55e64798-tmp\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.381307 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.381273    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbmv8\" (UniqueName: \"kubernetes.io/projected/b8a4d7e7-5ed2-4876-9851-72ac034f5f2c-kube-api-access-qbmv8\") pod \"multus-lsq54\" (UID: \"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c\") " pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.384037 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.384013    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqvb2\" (UniqueName: \"kubernetes.io/projected/112bace8-f3ae-4a77-aa75-a40214e6a48d-kube-api-access-mqvb2\") pod \"multus-additional-cni-plugins-587br\" (UID: \"112bace8-f3ae-4a77-aa75-a40214e6a48d\") " pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.384623 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.384593    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrt5h\" (UniqueName: \"kubernetes.io/projected/090239e9-6752-41f7-bb38-677b56132111-kube-api-access-wrt5h\") pod \"aws-ebs-csi-driver-node-twjz7\" (UID: \"090239e9-6752-41f7-bb38-677b56132111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.386401 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.386384    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxrpt\" (UniqueName: \"kubernetes.io/projected/b9f3b082-85fb-4e61-84eb-11697e1dbb76-kube-api-access-qxrpt\") pod \"node-resolver-bt4zv\" (UID: \"b9f3b082-85fb-4e61-84eb-11697e1dbb76\") " pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.389460 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.389440    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h77ws\" (UniqueName: \"kubernetes.io/projected/9cf4a098-9d50-4bf7-af7b-576e55e64798-kube-api-access-h77ws\") pod \"tuned-qpf9v\" (UID: \"9cf4a098-9d50-4bf7-af7b-576e55e64798\") " pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.476591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476563    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovnkube-config\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476591 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476595    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-env-overrides\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476619    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovnkube-script-lib\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476642    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-var-lib-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476687    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/313b0821-33f6-430d-b1c7-7b1723f5930c-host\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476709    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-systemd-units\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476728    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476756    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-kubelet\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476773    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-systemd\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476794    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-run-ovn-kubernetes\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.476818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476813    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-cni-bin\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476834    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476858    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-etc-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476886    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e880c852-03c2-4730-ae0e-26df38f98598-agent-certs\") pod \"konnectivity-agent-hvksz\" (UID: \"e880c852-03c2-4730-ae0e-26df38f98598\") " pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476917    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/313b0821-33f6-430d-b1c7-7b1723f5930c-serviceca\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476974    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb62578b-c505-45b0-9606-8f94982ec824-iptables-alerter-script\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.476998    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-node-log\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477030    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e880c852-03c2-4730-ae0e-26df38f98598-konnectivity-ca\") pod \"konnectivity-agent-hvksz\" (UID: \"e880c852-03c2-4730-ae0e-26df38f98598\") " pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477051    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41653b25-2617-4640-a58f-1853517ff74d-kubelet-config\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477083    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqvxg\" (UniqueName: \"kubernetes.io/projected/313b0821-33f6-430d-b1c7-7b1723f5930c-kube-api-access-mqvxg\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477127    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477162    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-ovn\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477170    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-env-overrides\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477210 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477181    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb62578b-c505-45b0-9606-8f94982ec824-host-slash\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477211    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovnkube-config\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477206    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovn-node-metrics-cert\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477252    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-etc-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477257    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477305    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41653b25-2617-4640-a58f-1853517ff74d-dbus\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477332    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-run-netns\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477349    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-cni-netd\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477370    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmdp6\" (UniqueName: \"kubernetes.io/projected/b53caea2-cf8f-4fe3-9b37-defacc71a768-kube-api-access-jmdp6\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477388    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq7zs\" (UniqueName: \"kubernetes.io/projected/fb62578b-c505-45b0-9606-8f94982ec824-kube-api-access-rq7zs\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477403    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-log-socket\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477418    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477440    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ntpj\" (UniqueName: \"kubernetes.io/projected/f6881e29-762a-4296-8c61-1b1774a6577a-kube-api-access-8ntpj\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477463    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-slash\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477547    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-slash\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477660    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.477796 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.477712    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-log-socket\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.477807    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.477874    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs podName:f6881e29-762a-4296-8c61-1b1774a6577a nodeName:}" failed. No retries permitted until 2026-07-02 19:56:30.977846911 +0000 UTC m=+3.125615669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs") pod "network-metrics-daemon-d2pws" (UID: "f6881e29-762a-4296-8c61-1b1774a6577a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478054    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41653b25-2617-4640-a58f-1853517ff74d-dbus\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478082    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovnkube-script-lib\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478085    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-run-netns\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478112    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-cni-netd\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478125    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-var-lib-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478152    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/313b0821-33f6-430d-b1c7-7b1723f5930c-host\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478154    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-node-log\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478179    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-systemd\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478180    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e880c852-03c2-4730-ae0e-26df38f98598-konnectivity-ca\") pod \"konnectivity-agent-hvksz\" (UID: \"e880c852-03c2-4730-ae0e-26df38f98598\") " pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478204    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-systemd-units\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478184    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-run-ovn-kubernetes\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478232    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-openvswitch\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478244    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-cni-bin\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478266    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41653b25-2617-4640-a58f-1853517ff74d-kubelet-config\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.478552 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478409    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/313b0821-33f6-430d-b1c7-7b1723f5930c-serviceca\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.479271 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478890    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb62578b-c505-45b0-9606-8f94982ec824-iptables-alerter-script\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.479271 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478953    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-run-ovn\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.479271 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.478997    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b53caea2-cf8f-4fe3-9b37-defacc71a768-host-kubelet\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.479271 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.479066    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:30.479271 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.479104    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret podName:41653b25-2617-4640-a58f-1853517ff74d nodeName:}" failed. No retries permitted until 2026-07-02 19:56:30.97909439 +0000 UTC m=+3.126863145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret") pod "global-pull-secret-syncer-llksn" (UID: "41653b25-2617-4640-a58f-1853517ff74d") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:30.479271 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.479140    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb62578b-c505-45b0-9606-8f94982ec824-host-slash\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.479995 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.479969    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b53caea2-cf8f-4fe3-9b37-defacc71a768-ovn-node-metrics-cert\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.480851 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.480826    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e880c852-03c2-4730-ae0e-26df38f98598-agent-certs\") pod \"konnectivity-agent-hvksz\" (UID: \"e880c852-03c2-4730-ae0e-26df38f98598\") " pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.487570 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.487549    2536 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 19:56:30.487694 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.487678    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:56:30.487828 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.487705    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:56:30.487828 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.487717    2536 projected.go:196] Error preparing data for projected volume kube-api-access-4wrbv for pod openshift-network-diagnostics/network-check-target-7mf8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:30.487828 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.487766    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv podName:cdedba5b-0c4b-453d-b5b9-30da82f6fef3 nodeName:}" failed. No retries permitted until 2026-07-02 19:56:30.987752928 +0000 UTC m=+3.135521683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrbv" (UniqueName: "kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv") pod "network-check-target-7mf8w" (UID: "cdedba5b-0c4b-453d-b5b9-30da82f6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:30.489788 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.489764    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq7zs\" (UniqueName: \"kubernetes.io/projected/fb62578b-c505-45b0-9606-8f94982ec824-kube-api-access-rq7zs\") pod \"iptables-alerter-ss72c\" (UID: \"fb62578b-c505-45b0-9606-8f94982ec824\") " pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.489889 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.489813    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ntpj\" (UniqueName: \"kubernetes.io/projected/f6881e29-762a-4296-8c61-1b1774a6577a-kube-api-access-8ntpj\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:30.490852 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.490835    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmdp6\" (UniqueName: \"kubernetes.io/projected/b53caea2-cf8f-4fe3-9b37-defacc71a768-kube-api-access-jmdp6\") pod \"ovnkube-node-8dzn4\" (UID: \"b53caea2-cf8f-4fe3-9b37-defacc71a768\") " pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.490960 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.490944    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqvxg\" (UniqueName: \"kubernetes.io/projected/313b0821-33f6-430d-b1c7-7b1723f5930c-kube-api-access-mqvxg\") pod \"node-ca-xnztd\" (UID: \"313b0821-33f6-430d-b1c7-7b1723f5930c\") " pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.566239 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.566167    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-587br"
Jul 02 19:56:30.572142 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.572126    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lsq54"
Jul 02 19:56:30.580697 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.580677    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7"
Jul 02 19:56:30.585888 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.585871    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qpf9v"
Jul 02 19:56:30.593324 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.593306    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bt4zv"
Jul 02 19:56:30.600819 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.600800    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:30.607262 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.607241    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ss72c"
Jul 02 19:56:30.615654 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.615638    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:30.628801 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.628784    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xnztd"
Jul 02 19:56:30.980486 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.980413    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:30.980486 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:30.980482    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:30.980684 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.980552    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:30.980684 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.980585    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:30.980684 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.980621    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret podName:41653b25-2617-4640-a58f-1853517ff74d nodeName:}" failed. No retries permitted until 2026-07-02 19:56:31.980600529 +0000 UTC m=+4.128369298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret") pod "global-pull-secret-syncer-llksn" (UID: "41653b25-2617-4640-a58f-1853517ff74d") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:30.980684 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:30.980636    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs podName:f6881e29-762a-4296-8c61-1b1774a6577a nodeName:}" failed. No retries permitted until 2026-07-02 19:56:31.980629446 +0000 UTC m=+4.128398204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs") pod "network-metrics-daemon-d2pws" (UID: "f6881e29-762a-4296-8c61-1b1774a6577a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:31.080990 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.080957    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:31.081151 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.081095    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:56:31.081151 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.081117    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:56:31.081151 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.081131    2536 projected.go:196] Error preparing data for projected volume kube-api-access-4wrbv for pod openshift-network-diagnostics/network-check-target-7mf8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:31.081305 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.081183    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv podName:cdedba5b-0c4b-453d-b5b9-30da82f6fef3 nodeName:}" failed. No retries permitted until 2026-07-02 19:56:32.081167235 +0000 UTC m=+4.228935994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrbv" (UniqueName: "kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv") pod "network-check-target-7mf8w" (UID: "cdedba5b-0c4b-453d-b5b9-30da82f6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:31.249072 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.249049    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112bace8_f3ae_4a77_aa75_a40214e6a48d.slice/crio-f8ca987613db326b50c14fab705e004322f3d124f758841126af1578d21b8b8a WatchSource:0}: Error finding container f8ca987613db326b50c14fab705e004322f3d124f758841126af1578d21b8b8a: Status 404 returned error can't find the container with id f8ca987613db326b50c14fab705e004322f3d124f758841126af1578d21b8b8a
Jul 02 19:56:31.254304 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.253044    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod090239e9_6752_41f7_bb38_677b56132111.slice/crio-dc6a599cd49b100b1dbe27d9ee308ac6db559e294a2a25b4c6e8b19c249e4175 WatchSource:0}: Error finding container dc6a599cd49b100b1dbe27d9ee308ac6db559e294a2a25b4c6e8b19c249e4175: Status 404 returned error can't find the container with id dc6a599cd49b100b1dbe27d9ee308ac6db559e294a2a25b4c6e8b19c249e4175
Jul 02 19:56:31.254667 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.254649    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb62578b_c505_45b0_9606_8f94982ec824.slice/crio-0b0fef8bd739acb2fef2b9849d9aad6e60494320e5724d4d534c0bc4799d8f59 WatchSource:0}: Error finding container 0b0fef8bd739acb2fef2b9849d9aad6e60494320e5724d4d534c0bc4799d8f59: Status 404 returned error can't find the container with id 0b0fef8bd739acb2fef2b9849d9aad6e60494320e5724d4d534c0bc4799d8f59
Jul 02 19:56:31.255540 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.255517    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a4d7e7_5ed2_4876_9851_72ac034f5f2c.slice/crio-e35d767d9d8bb618ffdd477f4d75bfd20d85ac84ba0cb138de22606dddd1f868 WatchSource:0}: Error finding container e35d767d9d8bb618ffdd477f4d75bfd20d85ac84ba0cb138de22606dddd1f868: Status 404 returned error can't find the container with id e35d767d9d8bb618ffdd477f4d75bfd20d85ac84ba0cb138de22606dddd1f868
Jul 02 19:56:31.256267 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.256126    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53caea2_cf8f_4fe3_9b37_defacc71a768.slice/crio-d3fc3e351520ff9081d4dd93578889ba152ed312bb9706299319cc9daafba98c WatchSource:0}: Error finding container d3fc3e351520ff9081d4dd93578889ba152ed312bb9706299319cc9daafba98c: Status 404 returned error can't find the container with id d3fc3e351520ff9081d4dd93578889ba152ed312bb9706299319cc9daafba98c
Jul 02 19:56:31.256794 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.256774    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf4a098_9d50_4bf7_af7b_576e55e64798.slice/crio-1697a2703d9457ae7e8247004a6ff62173add07edd413a812e46d6f3d3bdf66f WatchSource:0}: Error finding container 1697a2703d9457ae7e8247004a6ff62173add07edd413a812e46d6f3d3bdf66f: Status 404 returned error can't find the container with id 1697a2703d9457ae7e8247004a6ff62173add07edd413a812e46d6f3d3bdf66f
Jul 02 19:56:31.258794 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.258778    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode880c852_03c2_4730_ae0e_26df38f98598.slice/crio-41f5be18df5545a5aa21a3cddcce4a3c0615b0d6fc654e5df2369277e9c3d928 WatchSource:0}: Error finding container 41f5be18df5545a5aa21a3cddcce4a3c0615b0d6fc654e5df2369277e9c3d928: Status 404 returned error can't find the container with id 41f5be18df5545a5aa21a3cddcce4a3c0615b0d6fc654e5df2369277e9c3d928
Jul 02 19:56:31.260743 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.260439    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313b0821_33f6_430d_b1c7_7b1723f5930c.slice/crio-ad9c4e1ebc28307ff6d87a81b598d37bb34512d6849a52f96d852b970dfb90fe WatchSource:0}: Error finding container ad9c4e1ebc28307ff6d87a81b598d37bb34512d6849a52f96d852b970dfb90fe: Status 404 returned error can't find the container with id ad9c4e1ebc28307ff6d87a81b598d37bb34512d6849a52f96d852b970dfb90fe
Jul 02 19:56:31.261347 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:56:31.261256    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f3b082_85fb_4e61_84eb_11697e1dbb76.slice/crio-4116e6bfb0f3d1a1f06264aee47b5241fed52c2ae5f6110f72adad01df83a420 WatchSource:0}: Error finding container 4116e6bfb0f3d1a1f06264aee47b5241fed52c2ae5f6110f72adad01df83a420: Status 404 returned error can't find the container with id 4116e6bfb0f3d1a1f06264aee47b5241fed52c2ae5f6110f72adad01df83a420
Jul 02 19:56:31.348016 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.347886    2536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:51:30 +0000 UTC" deadline="2028-04-12 16:48:51.251959312 +0000 UTC"
Jul 02 19:56:31.348016 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.348015    2536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15596h52m19.903948109s"
Jul 02 19:56:31.370930 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.370901    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xnztd" event={"ID":"313b0821-33f6-430d-b1c7-7b1723f5930c","Type":"ContainerStarted","Data":"ad9c4e1ebc28307ff6d87a81b598d37bb34512d6849a52f96d852b970dfb90fe"}
Jul 02 19:56:31.371911 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.371888    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hvksz" event={"ID":"e880c852-03c2-4730-ae0e-26df38f98598","Type":"ContainerStarted","Data":"41f5be18df5545a5aa21a3cddcce4a3c0615b0d6fc654e5df2369277e9c3d928"}
Jul 02 19:56:31.372923 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.372906    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsq54" event={"ID":"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c","Type":"ContainerStarted","Data":"e35d767d9d8bb618ffdd477f4d75bfd20d85ac84ba0cb138de22606dddd1f868"}
Jul 02 19:56:31.374532 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.374511    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ss72c" event={"ID":"fb62578b-c505-45b0-9606-8f94982ec824","Type":"ContainerStarted","Data":"0b0fef8bd739acb2fef2b9849d9aad6e60494320e5724d4d534c0bc4799d8f59"}
Jul 02 19:56:31.375691 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.375671    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerStarted","Data":"f8ca987613db326b50c14fab705e004322f3d124f758841126af1578d21b8b8a"}
Jul 02 19:56:31.376734 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.376709    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bt4zv" event={"ID":"b9f3b082-85fb-4e61-84eb-11697e1dbb76","Type":"ContainerStarted","Data":"4116e6bfb0f3d1a1f06264aee47b5241fed52c2ae5f6110f72adad01df83a420"}
Jul 02 19:56:31.377704 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.377684    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qpf9v" event={"ID":"9cf4a098-9d50-4bf7-af7b-576e55e64798","Type":"ContainerStarted","Data":"1697a2703d9457ae7e8247004a6ff62173add07edd413a812e46d6f3d3bdf66f"}
Jul 02 19:56:31.380148 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.379188    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"d3fc3e351520ff9081d4dd93578889ba152ed312bb9706299319cc9daafba98c"}
Jul 02 19:56:31.381553 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.381536    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7" event={"ID":"090239e9-6752-41f7-bb38-677b56132111","Type":"ContainerStarted","Data":"dc6a599cd49b100b1dbe27d9ee308ac6db559e294a2a25b4c6e8b19c249e4175"}
Jul 02 19:56:31.987578 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.987485    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:31.987735 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:31.987585    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:31.987735 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.987648    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:31.987735 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.987667    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:31.987735 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.987714    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs podName:f6881e29-762a-4296-8c61-1b1774a6577a nodeName:}" failed. No retries permitted until 2026-07-02 19:56:33.98769633 +0000 UTC m=+6.135465087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs") pod "network-metrics-daemon-d2pws" (UID: "f6881e29-762a-4296-8c61-1b1774a6577a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:31.987735 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:31.987734    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret podName:41653b25-2617-4640-a58f-1853517ff74d nodeName:}" failed. No retries permitted until 2026-07-02 19:56:33.987724429 +0000 UTC m=+6.135493202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret") pod "global-pull-secret-syncer-llksn" (UID: "41653b25-2617-4640-a58f-1853517ff74d") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:32.088684 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.088642    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:32.088860 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:32.088834    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:56:32.088860 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:32.088853    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:56:32.088976 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:32.088866    2536 projected.go:196] Error preparing data for projected volume kube-api-access-4wrbv for pod openshift-network-diagnostics/network-check-target-7mf8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:32.088976 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:32.088923    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv podName:cdedba5b-0c4b-453d-b5b9-30da82f6fef3 nodeName:}" failed. No retries permitted until 2026-07-02 19:56:34.088905338 +0000 UTC m=+6.236674112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrbv" (UniqueName: "kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv") pod "network-check-target-7mf8w" (UID: "cdedba5b-0c4b-453d-b5b9-30da82f6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:32.348468 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.348363    2536 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:51:30 +0000 UTC" deadline="2027-12-11 19:33:20.751879737 +0000 UTC"
Jul 02 19:56:32.348468 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.348425    2536 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12647h36m48.403460157s"
Jul 02 19:56:32.371446 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.371416    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:32.371573 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:32.371540    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:32.371990 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.371918    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:32.372086 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:32.372023    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:32.372141 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.372090    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:32.372238 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:32.372171    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:32.408775 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.408005    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal" event={"ID":"6a63f3713d25ef95e553ae4a5cda922f","Type":"ContainerStarted","Data":"acf7a853a6c178167e95ff65b4b601666e2121e624d2a605f2029eb1cff8b119"}
Jul 02 19:56:32.418236 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.417622    2536 generic.go:356] "Generic (PLEG): container finished" podID="de540d23a4bee0040959efe51be6b698" containerID="af652da305d06c179c6ba1a31f27252aeb991c7e33c2e0d2651d74bfe08d54b6" exitCode=0
Jul 02 19:56:32.418236 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.417669    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal" event={"ID":"de540d23a4bee0040959efe51be6b698","Type":"ContainerDied","Data":"af652da305d06c179c6ba1a31f27252aeb991c7e33c2e0d2651d74bfe08d54b6"}
Jul 02 19:56:32.424673 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:32.424452    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-101.ec2.internal" podStartSLOduration=3.424432454 podStartE2EDuration="3.424432454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:32.423261996 +0000 UTC m=+4.571030775" watchObservedRunningTime="2026-07-02 19:56:32.424432454 +0000 UTC m=+4.572201236"
Jul 02 19:56:33.443786 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:33.443707    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal" event={"ID":"de540d23a4bee0040959efe51be6b698","Type":"ContainerStarted","Data":"b8483ca4f8faf65ffa54fee6a541771a9a179804caf1586f77f5277d8219bac9"}
Jul 02 19:56:33.460451 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:33.459478    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-101.ec2.internal" podStartSLOduration=4.459457434 podStartE2EDuration="4.459457434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:33.459162558 +0000 UTC m=+5.606931338" watchObservedRunningTime="2026-07-02 19:56:33.459457434 +0000 UTC m=+5.607226217"
Jul 02 19:56:34.006085 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:34.006046    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:34.006249 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:34.006139    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:34.006349 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.006256    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:34.006349 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.006340    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret podName:41653b25-2617-4640-a58f-1853517ff74d nodeName:}" failed. No retries permitted until 2026-07-02 19:56:38.006320694 +0000 UTC m=+10.154089457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret") pod "global-pull-secret-syncer-llksn" (UID: "41653b25-2617-4640-a58f-1853517ff74d") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:34.006801 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.006780    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:34.006900 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.006833    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs podName:f6881e29-762a-4296-8c61-1b1774a6577a nodeName:}" failed. No retries permitted until 2026-07-02 19:56:38.006819456 +0000 UTC m=+10.154588219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs") pod "network-metrics-daemon-d2pws" (UID: "f6881e29-762a-4296-8c61-1b1774a6577a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:34.108175 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:34.107427    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:34.108175 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.107688    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:56:34.108175 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.107711    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:56:34.108175 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.107723    2536 projected.go:196] Error preparing data for projected volume kube-api-access-4wrbv for pod openshift-network-diagnostics/network-check-target-7mf8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:34.108175 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.107782    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv podName:cdedba5b-0c4b-453d-b5b9-30da82f6fef3 nodeName:}" failed. No retries permitted until 2026-07-02 19:56:38.107764547 +0000 UTC m=+10.255533308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrbv" (UniqueName: "kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv") pod "network-check-target-7mf8w" (UID: "cdedba5b-0c4b-453d-b5b9-30da82f6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:34.363763 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:34.363685    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:34.363929 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.363838    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:34.364225 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:34.364204    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:34.364340 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.364318    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:34.364455 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:34.364397    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:34.364507 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:34.364487    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:36.365772 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:36.365744    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:36.366394 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:36.365890    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:36.366747 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:36.366726    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:36.366848 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:36.366828    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:36.366906 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:36.366896    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:36.366985 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:36.366967    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:38.040793 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:38.040749    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:38.041187 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:38.040816    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:38.041187 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.040947    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:38.041187 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.041003    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs podName:f6881e29-762a-4296-8c61-1b1774a6577a nodeName:}" failed. No retries permitted until 2026-07-02 19:56:46.040984314 +0000 UTC m=+18.188753087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs") pod "network-metrics-daemon-d2pws" (UID: "f6881e29-762a-4296-8c61-1b1774a6577a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:38.041461 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.041437    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:38.041552 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.041510    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret podName:41653b25-2617-4640-a58f-1853517ff74d nodeName:}" failed. No retries permitted until 2026-07-02 19:56:46.041492541 +0000 UTC m=+18.189261312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret") pod "global-pull-secret-syncer-llksn" (UID: "41653b25-2617-4640-a58f-1853517ff74d") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:38.141841 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:38.141810    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:38.142081 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.142038    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:56:38.142081 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.142065    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:56:38.142081 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.142078    2536 projected.go:196] Error preparing data for projected volume kube-api-access-4wrbv for pod openshift-network-diagnostics/network-check-target-7mf8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:38.142208 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.142134    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv podName:cdedba5b-0c4b-453d-b5b9-30da82f6fef3 nodeName:}" failed. No retries permitted until 2026-07-02 19:56:46.142114547 +0000 UTC m=+18.289883321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrbv" (UniqueName: "kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv") pod "network-check-target-7mf8w" (UID: "cdedba5b-0c4b-453d-b5b9-30da82f6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:38.364514 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:38.364442    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:38.364666 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:38.364452    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:38.364666 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.364559    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:38.364666 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.364633    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:38.364812 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:38.364672    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:38.364812 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:38.364731    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:40.363372 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:40.363336    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:40.363815 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:40.363343    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:40.363815 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:40.363475    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:40.363815 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:40.363343    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:40.363815 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:40.363517    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:40.363815 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:40.363578    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:42.363198 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:42.363168    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:42.363610 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:42.363169    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:42.363610 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:42.363304    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:42.363610 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:42.363305    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:42.363610 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:42.363390    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:42.363610 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:42.363468    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:44.363046 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:44.363016    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:44.363538 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:44.363016    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:44.363538 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:44.363181    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:44.363538 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:44.363233    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:44.363538 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:44.363023    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:44.363538 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:44.363366    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:46.104315 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:46.104258    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:46.104764 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:46.104366    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:46.104764 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.104415    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:46.104764 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.104472    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:46.104764 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.104485    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs podName:f6881e29-762a-4296-8c61-1b1774a6577a nodeName:}" failed. No retries permitted until 2026-07-02 19:57:02.104466168 +0000 UTC m=+34.252234946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs") pod "network-metrics-daemon-d2pws" (UID: "f6881e29-762a-4296-8c61-1b1774a6577a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:56:46.104764 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.104504    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret podName:41653b25-2617-4640-a58f-1853517ff74d nodeName:}" failed. No retries permitted until 2026-07-02 19:57:02.104493988 +0000 UTC m=+34.252262744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret") pod "global-pull-secret-syncer-llksn" (UID: "41653b25-2617-4640-a58f-1853517ff74d") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:56:46.205443 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:46.205403    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:46.205623 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.205578    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:56:46.205623 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.205598    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:56:46.205623 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.205607    2536 projected.go:196] Error preparing data for projected volume kube-api-access-4wrbv for pod openshift-network-diagnostics/network-check-target-7mf8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:46.205739 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.205658    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv podName:cdedba5b-0c4b-453d-b5b9-30da82f6fef3 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:02.205642619 +0000 UTC m=+34.353411393 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrbv" (UniqueName: "kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv") pod "network-check-target-7mf8w" (UID: "cdedba5b-0c4b-453d-b5b9-30da82f6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:56:46.363778 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:46.363700    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:46.363922 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.363798    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:46.363922 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:46.363700    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:46.363922 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:46.363845    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:46.364096 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.363945    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:46.364096 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:46.364032    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:48.364511 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:48.364480    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:48.364913 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:48.364590    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:48.364913 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:48.364600    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:48.364913 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:48.364703    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:48.364913 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:48.364730    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:48.364913 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:48.364770    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:50.363648 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:50.363610    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:50.364202 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:50.363617    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:50.364202 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:50.363755    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:50.364202 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:50.363618    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:50.364202 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:50.363828    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:50.364202 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:50.363895    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:51.485254 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.484953    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7" event={"ID":"090239e9-6752-41f7-bb38-677b56132111","Type":"ContainerStarted","Data":"d7fa0a27ebfe9ab2ff9727355be311cb86b8217779367356247a219a2c851322"}
Jul 02 19:56:51.486676 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.486652    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xnztd" event={"ID":"313b0821-33f6-430d-b1c7-7b1723f5930c","Type":"ContainerStarted","Data":"eec9c948f9477cf18655a14a9066b5dc5799a6c976567031e3cf8630cfb73489"}
Jul 02 19:56:51.488159 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.488124    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hvksz" event={"ID":"e880c852-03c2-4730-ae0e-26df38f98598","Type":"ContainerStarted","Data":"a70161fe356c88a59f88b146cb2f64940d6988461b1df3c6df1768dd26b50719"}
Jul 02 19:56:51.490063 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.490042    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lsq54" event={"ID":"b8a4d7e7-5ed2-4876-9851-72ac034f5f2c","Type":"ContainerStarted","Data":"4a8dc8e8729d0a539120249a6e35cb49be3455328092917a6badd39f23d73f09"}
Jul 02 19:56:51.491360 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.491339    2536 generic.go:356] "Generic (PLEG): container finished" podID="112bace8-f3ae-4a77-aa75-a40214e6a48d" containerID="f85ad04ba44586816cbdde3e2be6b3b6e0cb637d21993fdccef514be6b5477f3" exitCode=0
Jul 02 19:56:51.491595 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.491403    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerDied","Data":"f85ad04ba44586816cbdde3e2be6b3b6e0cb637d21993fdccef514be6b5477f3"}
Jul 02 19:56:51.492822 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.492789    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bt4zv" event={"ID":"b9f3b082-85fb-4e61-84eb-11697e1dbb76","Type":"ContainerStarted","Data":"4e8346252736a834709cc42f85676e1760c27a1271ac12f51577b1e076dc33df"}
Jul 02 19:56:51.494173 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.494152    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qpf9v" event={"ID":"9cf4a098-9d50-4bf7-af7b-576e55e64798","Type":"ContainerStarted","Data":"adf7c636dbab23a5a7e343d8cc16649a70d1007f33039c05b41ce06c6c235216"}
Jul 02 19:56:51.496279 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.496259    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"90fa621ae77bbc803ec42cb2b4cd03f3a5f00f4c825272d2e74611be60a735f5"}
Jul 02 19:56:51.496417 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.496370    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"a407595d30859ac5681e80f488726995229dc1569e63a4797f98b334805ac006"}
Jul 02 19:56:51.496463 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.496429    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"558b2737c5b1bf3a930c2d170fc80b04eb8c84fa4f76f5c31fb51feb327d05da"}
Jul 02 19:56:51.500961 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.500924    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xnztd" podStartSLOduration=3.759988249 podStartE2EDuration="23.500911666s" totalImagesPullingTime="19.740923417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:51.500374135 +0000 UTC m=+23.648142933" watchObservedRunningTime="2026-07-02 19:56:51.500911666 +0000 UTC m=+23.648680445"
Jul 02 19:56:51.524224 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.524186    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lsq54" podStartSLOduration=3.723555946 podStartE2EDuration="23.524173733s" totalImagesPullingTime="19.800617787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:51.524163033 +0000 UTC m=+23.671931812" watchObservedRunningTime="2026-07-02 19:56:51.524173733 +0000 UTC m=+23.671942512"
Jul 02 19:56:51.554900 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.554863    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hvksz" podStartSLOduration=3.792423187 podStartE2EDuration="23.554850922s" totalImagesPullingTime="19.762427735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:51.554590677 +0000 UTC m=+23.702359470" watchObservedRunningTime="2026-07-02 19:56:51.554850922 +0000 UTC m=+23.702619701"
Jul 02 19:56:51.555178 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.555160    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bt4zv" podStartSLOduration=3.787328349 podStartE2EDuration="23.555156578s" totalImagesPullingTime="19.767828229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:51.537875259 +0000 UTC m=+23.685644039" watchObservedRunningTime="2026-07-02 19:56:51.555156578 +0000 UTC m=+23.702925410"
Jul 02 19:56:51.598465 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:51.598414    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qpf9v" podStartSLOduration=3.84351505 podStartE2EDuration="23.598400488s" totalImagesPullingTime="19.754885438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:51.598215333 +0000 UTC m=+23.745984321" watchObservedRunningTime="2026-07-02 19:56:51.598400488 +0000 UTC m=+23.746169281"
Jul 02 19:56:52.364664 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.364635    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:52.364828 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:52.364754    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:52.365083 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.365069    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:52.365150 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:52.365136    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:52.365189 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.365180    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:52.365254 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:52.365242    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:52.484468 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.484439    2536 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 19:56:52.500246 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.500217    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ss72c" event={"ID":"fb62578b-c505-45b0-9606-8f94982ec824","Type":"ContainerStarted","Data":"7d843b3ad585bf834cf8fa679b36eb3c89b608616a2eb0d3ffcade0afeb751db"}
Jul 02 19:56:52.504083 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.504058    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"0db26e6c80024412971e108b6937359435c661d7116ed17a3e03f6c5f5b7081e"}
Jul 02 19:56:52.504207 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.504134    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"0ab1be4679234bb126d0f6c5c7050c9cb0a9f7494818db08415cf68eb1420a0e"}
Jul 02 19:56:52.504207 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.504179    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"025c283dc187486a3e50f62b336e0549f0b99c48792147b7ee2bdb8e229fe98c"}
Jul 02 19:56:52.506265 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.506243    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7" event={"ID":"090239e9-6752-41f7-bb38-677b56132111","Type":"ContainerStarted","Data":"75f50496ab088b0adc4f58a93d19e60e4f23cfeecfa97bf21445b3d9c264caa3"}
Jul 02 19:56:52.515453 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.515393    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ss72c" podStartSLOduration=4.747839124 podStartE2EDuration="24.51537642s" totalImagesPullingTime="19.767537296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:52.514590497 +0000 UTC m=+24.662359294" watchObservedRunningTime="2026-07-02 19:56:52.51537642 +0000 UTC m=+24.663145204"
Jul 02 19:56:52.779747 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.779715    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:52.780584 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:52.780560    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:53.337596 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:53.337503    2536 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T19:56:52.484461673Z","UUID":"10e4797c-268c-4f11-b333-5ba193e412b5","Handler":null,"Name":"","Endpoint":""}
Jul 02 19:56:53.339038 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:53.339011    2536 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 19:56:53.339161 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:53.339059    2536 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 19:56:53.519195 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:53.519174    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:53.519589 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:53.519571    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hvksz"
Jul 02 19:56:54.363488 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:54.363317    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:54.363661 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:54.363317    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:54.363661 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:54.363582    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:54.363781 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:54.363324    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:54.363781 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:54.363676    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:54.363781 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:54.363756    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:54.514833 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:54.514796    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"1df6074d1395eca9839cd39408d442673d49dfbd11ab956d779b1688eec3b9bd"}
Jul 02 19:56:54.517126 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:54.517098    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7" event={"ID":"090239e9-6752-41f7-bb38-677b56132111","Type":"ContainerStarted","Data":"de3b9142735140ee8350ea4c6030c5b06046949af87a90e284783e2a158d6549"}
Jul 02 19:56:54.536748 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:54.536682    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-twjz7" podStartSLOduration=4.708164427 podStartE2EDuration="26.53666681s" totalImagesPullingTime="21.828502383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:54.535401125 +0000 UTC m=+26.683169895" watchObservedRunningTime="2026-07-02 19:56:54.53666681 +0000 UTC m=+26.684435589"
Jul 02 19:56:56.363794 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.363563    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:56.364660 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.363564    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:56.364660 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:56.363833    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:56.364660 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.363567    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:56.364660 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:56.363884    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:56.364660 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:56.363981    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:56.524895 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.524852    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" event={"ID":"b53caea2-cf8f-4fe3-9b37-defacc71a768","Type":"ContainerStarted","Data":"a0fd841cde3db2ef1161187f9cf32066d47262ff0fd2af024222fae8063fb5a2"}
Jul 02 19:56:56.525094 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.525074    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:56.525175 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.525149    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:56.525213 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.525193    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:56.527012 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.526985    2536 generic.go:356] "Generic (PLEG): container finished" podID="112bace8-f3ae-4a77-aa75-a40214e6a48d" containerID="72994b7d6800302ea73647de6dc2407fcd1fee194d770750e6009595519fb0ed" exitCode=0
Jul 02 19:56:56.527131 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.527031    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerDied","Data":"72994b7d6800302ea73647de6dc2407fcd1fee194d770750e6009595519fb0ed"}
Jul 02 19:56:56.541194 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.541173    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:56.544093 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.544076    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:56:56.554184 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:56.554147    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4" podStartSLOduration=8.691035751 podStartE2EDuration="28.554134531s" totalImagesPullingTime="19.86309878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:56:56.552336419 +0000 UTC m=+28.700105216" watchObservedRunningTime="2026-07-02 19:56:56.554134531 +0000 UTC m=+28.701903310"
Jul 02 19:56:57.830949 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:57.830790    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7mf8w"]
Jul 02 19:56:57.831324 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:57.831064    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:57.831324 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:57.831152    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:56:57.833983 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:57.833966    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llksn"]
Jul 02 19:56:57.834071 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:57.834060    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:57.834163 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:57.834141    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:57.834966 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:57.834941    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d2pws"]
Jul 02 19:56:57.835111 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:57.835047    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:57.835252 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:57.835130    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:58.533218 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:58.533189    2536 generic.go:356] "Generic (PLEG): container finished" podID="112bace8-f3ae-4a77-aa75-a40214e6a48d" containerID="59e7b5494d6fa1086931a38082a4489119fc4a2113d6a35bbe60cb8a5d6e01ab" exitCode=0
Jul 02 19:56:58.533518 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:58.533275    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerDied","Data":"59e7b5494d6fa1086931a38082a4489119fc4a2113d6a35bbe60cb8a5d6e01ab"}
Jul 02 19:56:59.363540 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:59.363509    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:56:59.363979 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:59.363602    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:56:59.363979 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:59.363611    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:56:59.363979 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:59.363702    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:56:59.363979 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:56:59.363737    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:56:59.363979 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:56:59.363778    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:57:00.539882 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:00.539853    2536 generic.go:356] "Generic (PLEG): container finished" podID="112bace8-f3ae-4a77-aa75-a40214e6a48d" containerID="41a175dbf45dd52b8111dccf24589b28b54764bac86649a62bdf297d465130da" exitCode=0
Jul 02 19:57:00.540328 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:00.539909    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerDied","Data":"41a175dbf45dd52b8111dccf24589b28b54764bac86649a62bdf297d465130da"}
Jul 02 19:57:01.363113 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:01.363081    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:57:01.363313 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:01.363202    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:01.363313 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:01.363210    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:57:01.363313 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:01.363237    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:57:01.363529 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:01.363348    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:57:01.363529 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:01.363451    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:57:02.121824 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:02.121789    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:57:02.122268 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:02.121872    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:57:02.122268 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.121908    2536 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:57:02.122268 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.121980    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs podName:f6881e29-762a-4296-8c61-1b1774a6577a nodeName:}" failed. No retries permitted until 2026-07-02 19:57:34.121961782 +0000 UTC m=+66.269730557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs") pod "network-metrics-daemon-d2pws" (UID: "f6881e29-762a-4296-8c61-1b1774a6577a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 19:57:02.122268 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.121987    2536 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 19:57:02.122268 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.122039    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret podName:41653b25-2617-4640-a58f-1853517ff74d nodeName:}" failed. No retries permitted until 2026-07-02 19:57:34.1220229 +0000 UTC m=+66.269791674 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret") pod "global-pull-secret-syncer-llksn" (UID: "41653b25-2617-4640-a58f-1853517ff74d") : object "kube-system"/"original-pull-secret" not registered
Jul 02 19:57:02.222470 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:02.222436    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:02.222740 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.222708    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 19:57:02.222740 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.222737    2536 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 19:57:02.222921 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.222750    2536 projected.go:196] Error preparing data for projected volume kube-api-access-4wrbv for pod openshift-network-diagnostics/network-check-target-7mf8w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:57:02.222921 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:02.222841    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv podName:cdedba5b-0c4b-453d-b5b9-30da82f6fef3 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:34.222795671 +0000 UTC m=+66.370564445 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrbv" (UniqueName: "kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv") pod "network-check-target-7mf8w" (UID: "cdedba5b-0c4b-453d-b5b9-30da82f6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 19:57:02.617630 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:02.617436    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:02.717683 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:02.717645    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:03.259147 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.259114    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:03.362772 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.362711    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:57:03.362772 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.362767    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:57:03.362955 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.362767    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:03.362955 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:03.362841    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llksn" podUID="41653b25-2617-4640-a58f-1853517ff74d"
Jul 02 19:57:03.362955 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:03.362919    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d2pws" podUID="f6881e29-762a-4296-8c61-1b1774a6577a"
Jul 02 19:57:03.363046 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:03.362974    2536 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7mf8w" podUID="cdedba5b-0c4b-453d-b5b9-30da82f6fef3"
Jul 02 19:57:03.393984 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.393959    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:03.647185 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.647158    2536 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-101.ec2.internal" event="NodeReady"
Jul 02 19:57:03.647346 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.647278    2536 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 19:57:03.763837 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.763808    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hwq8h"]
Jul 02 19:57:03.765649 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.765634    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:03.773353 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.773332    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-859cd577c6-cpbqw"]
Jul 02 19:57:03.774230 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.774212    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w4q8x\""
Jul 02 19:57:03.774334 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.774250    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 19:57:03.775001 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.774986    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.775512 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.775495    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 19:57:03.779786 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.779764    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 02 19:57:03.779883 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.779797    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j7xqq"]
Jul 02 19:57:03.780687 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.780013    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 02 19:57:03.781644 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.781630    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:03.781888 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.781866    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 02 19:57:03.786741 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.786721    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 19:57:03.786916 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.786897    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cg89q\""
Jul 02 19:57:03.787179 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.787164    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 19:57:03.787390 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.787373    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 19:57:03.787907 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.787891    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vrpnh\""
Jul 02 19:57:03.791417 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.791391    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 02 19:57:03.810303 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.810211    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j7xqq"]
Jul 02 19:57:03.812664 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.812623    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hwq8h"]
Jul 02 19:57:03.813358 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.813341    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-859cd577c6-cpbqw"]
Jul 02 19:57:03.937026 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.936984    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-installation-pull-secrets\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.937026 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937030    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d428d5e0-4fd2-40e1-8cd6-be097512e90c-cert\") pod \"ingress-canary-j7xqq\" (UID: \"d428d5e0-4fd2-40e1-8cd6-be097512e90c\") " pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:03.937253 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937057    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.937253 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937100    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw8jg\" (UniqueName: \"kubernetes.io/projected/d428d5e0-4fd2-40e1-8cd6-be097512e90c-kube-api-access-mw8jg\") pod \"ingress-canary-j7xqq\" (UID: \"d428d5e0-4fd2-40e1-8cd6-be097512e90c\") " pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:03.937253 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937143    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d625d727-ce77-4086-acd0-b6610616afa8-config-volume\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:03.937253 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937211    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d625d727-ce77-4086-acd0-b6610616afa8-tmp-dir\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:03.937253 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937237    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-image-registry-private-configuration\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.937536 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937271    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxr25\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-kube-api-access-fxr25\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.937536 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937343    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-trusted-ca\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.937536 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937376    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzcmf\" (UniqueName: \"kubernetes.io/projected/d625d727-ce77-4086-acd0-b6610616afa8-kube-api-access-tzcmf\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:03.937536 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937399    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-certificates\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.937536 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937434    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:03.937536 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937465    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-bound-sa-token\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:03.937536 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:03.937494    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-ca-trust-extracted\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038358 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038320    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxr25\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-kube-api-access-fxr25\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038358 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038369    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-trusted-ca\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038397    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzcmf\" (UniqueName: \"kubernetes.io/projected/d625d727-ce77-4086-acd0-b6610616afa8-kube-api-access-tzcmf\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038419    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-certificates\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038457    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038472    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-bound-sa-token\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038491    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-ca-trust-extracted\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038524    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-installation-pull-secrets\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038551    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d428d5e0-4fd2-40e1-8cd6-be097512e90c-cert\") pod \"ingress-canary-j7xqq\" (UID: \"d428d5e0-4fd2-40e1-8cd6-be097512e90c\") " pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038575    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.038584    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038605    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw8jg\" (UniqueName: \"kubernetes.io/projected/d428d5e0-4fd2-40e1-8cd6-be097512e90c-kube-api-access-mw8jg\") pod \"ingress-canary-j7xqq\" (UID: \"d428d5e0-4fd2-40e1-8cd6-be097512e90c\") " pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:04.038633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038628    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d625d727-ce77-4086-acd0-b6610616afa8-config-volume\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.039156 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.038655    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls podName:d625d727-ce77-4086-acd0-b6610616afa8 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:04.538635975 +0000 UTC m=+36.686404752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls") pod "dns-default-hwq8h" (UID: "d625d727-ce77-4086-acd0-b6610616afa8") : secret "dns-default-metrics-tls" not found
Jul 02 19:57:04.039156 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038716    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d625d727-ce77-4086-acd0-b6610616afa8-tmp-dir\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.039156 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.038744    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-image-registry-private-configuration\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.039156 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.038865    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:57:04.039156 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.038882    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-859cd577c6-cpbqw: secret "image-registry-tls" not found
Jul 02 19:57:04.039156 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.038928    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls podName:50dc66fb-0be2-44a0-93cd-2d9976d4b75f nodeName:}" failed. No retries permitted until 2026-07-02 19:57:04.538912994 +0000 UTC m=+36.686681752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls") pod "image-registry-859cd577c6-cpbqw" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f") : secret "image-registry-tls" not found
Jul 02 19:57:04.039156 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.039033    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-certificates\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.039541 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.039170    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d625d727-ce77-4086-acd0-b6610616afa8-config-volume\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.039541 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.039424    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-trusted-ca\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.039603 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.039545    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d625d727-ce77-4086-acd0-b6610616afa8-tmp-dir\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.039603 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.039552    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-ca-trust-extracted\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.043164 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.043131    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d428d5e0-4fd2-40e1-8cd6-be097512e90c-cert\") pod \"ingress-canary-j7xqq\" (UID: \"d428d5e0-4fd2-40e1-8cd6-be097512e90c\") " pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:04.043164 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.043138    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-image-registry-private-configuration\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.043341 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.043142    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-installation-pull-secrets\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.054526 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.054474    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw8jg\" (UniqueName: \"kubernetes.io/projected/d428d5e0-4fd2-40e1-8cd6-be097512e90c-kube-api-access-mw8jg\") pod \"ingress-canary-j7xqq\" (UID: \"d428d5e0-4fd2-40e1-8cd6-be097512e90c\") " pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:04.056410 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.056332    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzcmf\" (UniqueName: \"kubernetes.io/projected/d625d727-ce77-4086-acd0-b6610616afa8-kube-api-access-tzcmf\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.056573 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.056554    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxr25\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-kube-api-access-fxr25\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.057153 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.057133    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-bound-sa-token\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.089351 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.089274    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j7xqq"
Jul 02 19:57:04.120009 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.119979    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:04.218606 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.218573    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:04.544556 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.544524    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:04.544988 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:04.544588    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:04.544988 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.544700    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:57:04.544988 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.544772    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls podName:d625d727-ce77-4086-acd0-b6610616afa8 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:05.5447572 +0000 UTC m=+37.692525957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls") pod "dns-default-hwq8h" (UID: "d625d727-ce77-4086-acd0-b6610616afa8") : secret "dns-default-metrics-tls" not found
Jul 02 19:57:04.544988 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.544703    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:57:04.544988 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.544808    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-859cd577c6-cpbqw: secret "image-registry-tls" not found
Jul 02 19:57:04.544988 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:04.544862    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls podName:50dc66fb-0be2-44a0-93cd-2d9976d4b75f nodeName:}" failed. No retries permitted until 2026-07-02 19:57:05.544848062 +0000 UTC m=+37.692616819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls") pod "image-registry-859cd577c6-cpbqw" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f") : secret "image-registry-tls" not found
Jul 02 19:57:05.363405 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.363375    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:57:05.363557 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.363375    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:57:05.363652 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.363377    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:05.366326 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.366299    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 19:57:05.366453 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.366303    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tj5bt\""
Jul 02 19:57:05.367613 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.367381    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 19:57:05.367613 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.367389    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 19:57:05.367613 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.367415    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 19:57:05.367613 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.367504    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g6jwx\""
Jul 02 19:57:05.466279 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.466257    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j7xqq"]
Jul 02 19:57:05.470246 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:05.470222    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd428d5e0_4fd2_40e1_8cd6_be097512e90c.slice/crio-d3272bf899a673e4545b6a5eb94c2d854d5349052357688af23972d923af08fb WatchSource:0}: Error finding container d3272bf899a673e4545b6a5eb94c2d854d5349052357688af23972d923af08fb: Status 404 returned error can't find the container with id d3272bf899a673e4545b6a5eb94c2d854d5349052357688af23972d923af08fb
Jul 02 19:57:05.555386 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.554665    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:05.555386 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.554938    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:05.555386 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:05.554976    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:57:05.555386 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:05.555044    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls podName:d625d727-ce77-4086-acd0-b6610616afa8 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:07.555020079 +0000 UTC m=+39.702788858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls") pod "dns-default-hwq8h" (UID: "d625d727-ce77-4086-acd0-b6610616afa8") : secret "dns-default-metrics-tls" not found
Jul 02 19:57:05.556892 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:05.556506    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:57:05.556892 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:05.556524    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-859cd577c6-cpbqw: secret "image-registry-tls" not found
Jul 02 19:57:05.556892 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:05.556580    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls podName:50dc66fb-0be2-44a0-93cd-2d9976d4b75f nodeName:}" failed. No retries permitted until 2026-07-02 19:57:07.556564242 +0000 UTC m=+39.704333015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls") pod "image-registry-859cd577c6-cpbqw" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f") : secret "image-registry-tls" not found
Jul 02 19:57:05.556892 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.556741    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j7xqq" event={"ID":"d428d5e0-4fd2-40e1-8cd6-be097512e90c","Type":"ContainerStarted","Data":"d3272bf899a673e4545b6a5eb94c2d854d5349052357688af23972d923af08fb"}
Jul 02 19:57:05.560820 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:05.560799    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerStarted","Data":"e51e5dd813824547e4bc1725c5b79079953b516ec51599d9e5615dd042c5bba7"}
Jul 02 19:57:06.121752 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:06.121711    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:06.219695 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:06.219652    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:06.566384 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:06.566348    2536 generic.go:356] "Generic (PLEG): container finished" podID="112bace8-f3ae-4a77-aa75-a40214e6a48d" containerID="e51e5dd813824547e4bc1725c5b79079953b516ec51599d9e5615dd042c5bba7" exitCode=0
Jul 02 19:57:06.566837 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:06.566426    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerDied","Data":"e51e5dd813824547e4bc1725c5b79079953b516ec51599d9e5615dd042c5bba7"}
Jul 02 19:57:07.463324 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.463262    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"]
Jul 02 19:57:07.487880 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.487846    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"]
Jul 02 19:57:07.488037 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.487933    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:07.490256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.490235    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 02 19:57:07.490391 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.490347    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hj56r\""
Jul 02 19:57:07.491388 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.491368    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 02 19:57:07.568838 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.568811    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.568851    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.568920    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e2e2791-aba8-4f69-a931-de18d3887569-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.568956    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:07.568966    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:07.568984    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-859cd577c6-cpbqw: secret "image-registry-tls" not found
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:07.569035    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls podName:50dc66fb-0be2-44a0-93cd-2d9976d4b75f nodeName:}" failed. No retries permitted until 2026-07-02 19:57:11.569021625 +0000 UTC m=+43.716790382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls") pod "image-registry-859cd577c6-cpbqw" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f") : secret "image-registry-tls" not found
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:07.569059    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:57:07.569461 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:07.569120    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls podName:d625d727-ce77-4086-acd0-b6610616afa8 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:11.569107352 +0000 UTC m=+43.716876108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls") pod "dns-default-hwq8h" (UID: "d625d727-ce77-4086-acd0-b6610616afa8") : secret "dns-default-metrics-tls" not found
Jul 02 19:57:07.571330 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.571309    2536 generic.go:356] "Generic (PLEG): container finished" podID="112bace8-f3ae-4a77-aa75-a40214e6a48d" containerID="dbc22751917f2d94de894fb466abd8ac40452563aa13f66fe3d8ac31dbd1a07d" exitCode=0
Jul 02 19:57:07.571423 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.571356    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerDied","Data":"dbc22751917f2d94de894fb466abd8ac40452563aa13f66fe3d8ac31dbd1a07d"}
Jul 02 19:57:07.594602 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.594578    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:07.669496 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.669472    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e2e2791-aba8-4f69-a931-de18d3887569-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:07.669745 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.669598    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:07.669836 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:07.669806    2536 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 19:57:07.669906 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:07.669890    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert podName:3e2e2791-aba8-4f69-a931-de18d3887569 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:08.169870482 +0000 UTC m=+40.317639239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-9g79g" (UID: "3e2e2791-aba8-4f69-a931-de18d3887569") : secret "networking-console-plugin-cert" not found
Jul 02 19:57:07.670143 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.670122    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e2e2791-aba8-4f69-a931-de18d3887569-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:07.696401 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.696378    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:07.795039 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.795001    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:07.857940 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:07.857911    2536 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log" limit=1024
Jul 02 19:57:08.172917 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:08.172889    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:08.173075 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:08.173033    2536 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 19:57:08.173112 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:08.173104    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert podName:3e2e2791-aba8-4f69-a931-de18d3887569 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:09.173079509 +0000 UTC m=+41.320848287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-9g79g" (UID: "3e2e2791-aba8-4f69-a931-de18d3887569") : secret "networking-console-plugin-cert" not found
Jul 02 19:57:08.577216 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:08.577179    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-587br" event={"ID":"112bace8-f3ae-4a77-aa75-a40214e6a48d","Type":"ContainerStarted","Data":"96cda2f600de56a36112fd2bcc85df1c0e8ec1162070ca2db7f4d24bd80caf3e"}
Jul 02 19:57:08.578665 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:08.578642    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j7xqq" event={"ID":"d428d5e0-4fd2-40e1-8cd6-be097512e90c","Type":"ContainerStarted","Data":"db93d165731ff12ab518befc6ca9780700b4317789d6ca34e30fcb04abdbb06b"}
Jul 02 19:57:08.606799 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:08.606762    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-587br" podStartSLOduration=8.978477536 podStartE2EDuration="40.606748635s" totalImagesPullingTime="31.628271099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:08.605684103 +0000 UTC m=+40.753452900" watchObservedRunningTime="2026-07-02 19:57:08.606748635 +0000 UTC m=+40.754517414"
Jul 02 19:57:08.623542 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:08.623501    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j7xqq" podStartSLOduration=3.150337985 podStartE2EDuration="5.623489048s" totalImagesPullingTime="2.473151063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:57:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:08.622927061 +0000 UTC m=+40.770695838" watchObservedRunningTime="2026-07-02 19:57:08.623489048 +0000 UTC m=+40.771257827"
Jul 02 19:57:09.178066 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:09.178033    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:09.178231 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:09.178170    2536 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 19:57:09.178231 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:09.178226    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert podName:3e2e2791-aba8-4f69-a931-de18d3887569 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:11.178211653 +0000 UTC m=+43.325980411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-9g79g" (UID: "3e2e2791-aba8-4f69-a931-de18d3887569") : secret "networking-console-plugin-cert" not found
Jul 02 19:57:11.193068 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:11.193030    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:11.193568 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:11.193149    2536 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 19:57:11.193568 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:11.193199    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert podName:3e2e2791-aba8-4f69-a931-de18d3887569 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:15.19318635 +0000 UTC m=+47.340955107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-9g79g" (UID: "3e2e2791-aba8-4f69-a931-de18d3887569") : secret "networking-console-plugin-cert" not found
Jul 02 19:57:11.596336 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:11.596249    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:11.596336 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:11.596303    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:11.596533 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:11.596398    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:57:11.596533 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:11.596414    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:57:11.596533 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:11.596424    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-859cd577c6-cpbqw: secret "image-registry-tls" not found
Jul 02 19:57:11.596533 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:11.596455    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls podName:d625d727-ce77-4086-acd0-b6610616afa8 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:19.596440352 +0000 UTC m=+51.744209108 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls") pod "dns-default-hwq8h" (UID: "d625d727-ce77-4086-acd0-b6610616afa8") : secret "dns-default-metrics-tls" not found
Jul 02 19:57:11.596533 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:11.596469    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls podName:50dc66fb-0be2-44a0-93cd-2d9976d4b75f nodeName:}" failed. No retries permitted until 2026-07-02 19:57:19.596462906 +0000 UTC m=+51.744231663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls") pod "image-registry-859cd577c6-cpbqw" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f") : secret "image-registry-tls" not found
Jul 02 19:57:12.987022 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:12.986980    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"]
Jul 02 19:57:13.018429 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.018396    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"]
Jul 02 19:57:13.018587 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.018519    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.021154 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.021128    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 02 19:57:13.021279 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.021137    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 02 19:57:13.022111 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.022081    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 02 19:57:13.022230 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.022095    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-52g8f\""
Jul 02 19:57:13.022230 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.022097    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 02 19:57:13.034444 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.034426    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"]
Jul 02 19:57:13.034550 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.034495    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"]
Jul 02 19:57:13.034550 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.034524    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"]
Jul 02 19:57:13.034550 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.034541    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.036870 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.036851    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 02 19:57:13.056520 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.056496    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"]
Jul 02 19:57:13.056613 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.056592    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.061902 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.061880    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 02 19:57:13.062624 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.062607    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 02 19:57:13.062855 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.062840    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 02 19:57:13.062938 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.062923    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 02 19:57:13.106337 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106314    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-ca\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.106430 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106347    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e26d2a-71f9-4547-8172-0073388f3df8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.106430 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106365    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-tmp\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.106430 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106381    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjg4b\" (UniqueName: \"kubernetes.io/projected/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-kube-api-access-gjg4b\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.106526 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106476    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-hub\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.106526 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106515    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdnjl\" (UniqueName: \"kubernetes.io/projected/d6ca7589-1a14-4546-bf92-872b5ad95e53-kube-api-access-pdnjl\") pod \"managed-serviceaccount-addon-agent-66bc7446c8-d4pf5\" (UID: \"d6ca7589-1a14-4546-bf92-872b5ad95e53\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.106586 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106531    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.106586 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106553    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6k4l\" (UniqueName: \"kubernetes.io/projected/48e26d2a-71f9-4547-8172-0073388f3df8-kube-api-access-r6k4l\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.106586 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106572    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6ca7589-1a14-4546-bf92-872b5ad95e53-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66bc7446c8-d4pf5\" (UID: \"d6ca7589-1a14-4546-bf92-872b5ad95e53\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.107042 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.106635    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.107079 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.107070    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-klusterlet-config\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.208312 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.208270    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6k4l\" (UniqueName: \"kubernetes.io/projected/48e26d2a-71f9-4547-8172-0073388f3df8-kube-api-access-r6k4l\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.208401 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.208319    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6ca7589-1a14-4546-bf92-872b5ad95e53-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66bc7446c8-d4pf5\" (UID: \"d6ca7589-1a14-4546-bf92-872b5ad95e53\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.208401 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.208341    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.208401 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.208370    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-klusterlet-config\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.208765 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.208742    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-ca\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.209824 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.209793    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e26d2a-71f9-4547-8172-0073388f3df8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.209923 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.209849    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-tmp\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.209923 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.209878    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjg4b\" (UniqueName: \"kubernetes.io/projected/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-kube-api-access-gjg4b\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.210610 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.210548    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-tmp\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.210692 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.210610    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-hub\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.210692 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.210676    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdnjl\" (UniqueName: \"kubernetes.io/projected/d6ca7589-1a14-4546-bf92-872b5ad95e53-kube-api-access-pdnjl\") pod \"managed-serviceaccount-addon-agent-66bc7446c8-d4pf5\" (UID: \"d6ca7589-1a14-4546-bf92-872b5ad95e53\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.210809 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.210709    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.210809 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.210780    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/48e26d2a-71f9-4547-8172-0073388f3df8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.214822 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.213178    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-ca\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.214822 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.213590    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.214822 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.213701    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d6ca7589-1a14-4546-bf92-872b5ad95e53-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66bc7446c8-d4pf5\" (UID: \"d6ca7589-1a14-4546-bf92-872b5ad95e53\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.214822 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.213721    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-klusterlet-config\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.215727 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.215704    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.215811 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.215726    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/48e26d2a-71f9-4547-8172-0073388f3df8-hub\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.217425 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.217406    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6k4l\" (UniqueName: \"kubernetes.io/projected/48e26d2a-71f9-4547-8172-0073388f3df8-kube-api-access-r6k4l\") pod \"cluster-proxy-proxy-agent-7f5c9b87db-pg47m\" (UID: \"48e26d2a-71f9-4547-8172-0073388f3df8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.217730 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.217706    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjg4b\" (UniqueName: \"kubernetes.io/projected/7e344b84-428c-4e6a-9ae9-b69f98cd4bd4-kube-api-access-gjg4b\") pod \"klusterlet-addon-workmgr-787b76ffbd-mm4hq\" (UID: \"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.219781 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.219756    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdnjl\" (UniqueName: \"kubernetes.io/projected/d6ca7589-1a14-4546-bf92-872b5ad95e53-kube-api-access-pdnjl\") pod \"managed-serviceaccount-addon-agent-66bc7446c8-d4pf5\" (UID: \"d6ca7589-1a14-4546-bf92-872b5ad95e53\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.341843 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.341779    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"
Jul 02 19:57:13.347931 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.347910    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:13.364297 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.364259    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:13.547141 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.547110    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5"]
Jul 02 19:57:13.552060 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:13.552032    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6ca7589_1a14_4546_bf92_872b5ad95e53.slice/crio-a5e529932d9e5e7e943ae0f85bc75e8434c31e58acb18c6f7375ccc6a21f4fd3 WatchSource:0}: Error finding container a5e529932d9e5e7e943ae0f85bc75e8434c31e58acb18c6f7375ccc6a21f4fd3: Status 404 returned error can't find the container with id a5e529932d9e5e7e943ae0f85bc75e8434c31e58acb18c6f7375ccc6a21f4fd3
Jul 02 19:57:13.561881 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.561834    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"]
Jul 02 19:57:13.563612 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:13.563591    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e344b84_428c_4e6a_9ae9_b69f98cd4bd4.slice/crio-a0ac9adf14e15321f6a678a5e1a418386c5aae160cca0c5173de9b52708ae174 WatchSource:0}: Error finding container a0ac9adf14e15321f6a678a5e1a418386c5aae160cca0c5173de9b52708ae174: Status 404 returned error can't find the container with id a0ac9adf14e15321f6a678a5e1a418386c5aae160cca0c5173de9b52708ae174
Jul 02 19:57:13.575829 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.575808    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"]
Jul 02 19:57:13.578271 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:13.578244    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48e26d2a_71f9_4547_8172_0073388f3df8.slice/crio-b352012a2ec316a597441d2d474453678fb5e88af2a5108adaf6b4365861a203 WatchSource:0}: Error finding container b352012a2ec316a597441d2d474453678fb5e88af2a5108adaf6b4365861a203: Status 404 returned error can't find the container with id b352012a2ec316a597441d2d474453678fb5e88af2a5108adaf6b4365861a203
Jul 02 19:57:13.591045 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.591019    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq" event={"ID":"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4","Type":"ContainerStarted","Data":"a0ac9adf14e15321f6a678a5e1a418386c5aae160cca0c5173de9b52708ae174"}
Jul 02 19:57:13.592176 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.592122    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" event={"ID":"48e26d2a-71f9-4547-8172-0073388f3df8","Type":"ContainerStarted","Data":"b352012a2ec316a597441d2d474453678fb5e88af2a5108adaf6b4365861a203"}
Jul 02 19:57:13.593206 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:13.593186    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5" event={"ID":"d6ca7589-1a14-4546-bf92-872b5ad95e53","Type":"ContainerStarted","Data":"a5e529932d9e5e7e943ae0f85bc75e8434c31e58acb18c6f7375ccc6a21f4fd3"}
Jul 02 19:57:14.739759 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.739727    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zqcmd"]
Jul 02 19:57:14.756092 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.756065    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.760906 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.760452    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 19:57:14.762454 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.761903    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqcmd"]
Jul 02 19:57:14.763618 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.762868    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 19:57:14.763618 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.763005    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 19:57:14.763618 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.763171    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-57z4p\""
Jul 02 19:57:14.763618 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.763380    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 19:57:14.823691 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.823655    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9905aed5-7419-45c0-a28a-aedc1a674c29-crio-socket\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.823864 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.823721    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vxlc\" (UniqueName: \"kubernetes.io/projected/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-api-access-9vxlc\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.823864 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.823762    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.823864 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.823835    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.824057 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.823874    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9905aed5-7419-45c0-a28a-aedc1a674c29-data-volume\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.925221 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.925181    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.925405 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.925280    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.925405 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.925348    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9905aed5-7419-45c0-a28a-aedc1a674c29-data-volume\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.925511 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.925425    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9905aed5-7419-45c0-a28a-aedc1a674c29-crio-socket\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.925511 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.925463    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxlc\" (UniqueName: \"kubernetes.io/projected/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-api-access-9vxlc\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.925944 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:14.925896    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:14.926058 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:14.925971    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:15.425949281 +0000 UTC m=+47.573718041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:14.926058 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:14.926027    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:57:14.926171 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:14.926069    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:15.426057481 +0000 UTC m=+47.573826252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : configmap "kube-rbac-proxy" not found
Jul 02 19:57:14.926449 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.926428    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9905aed5-7419-45c0-a28a-aedc1a674c29-data-volume\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.926637 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.926619    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9905aed5-7419-45c0-a28a-aedc1a674c29-crio-socket\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:14.938131 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:14.938107    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vxlc\" (UniqueName: \"kubernetes.io/projected/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-api-access-9vxlc\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:15.229374 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:15.228735    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:15.229374 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:15.228932    2536 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 19:57:15.229374 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:15.229000    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert podName:3e2e2791-aba8-4f69-a931-de18d3887569 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:23.228980784 +0000 UTC m=+55.376749547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-9g79g" (UID: "3e2e2791-aba8-4f69-a931-de18d3887569") : secret "networking-console-plugin-cert" not found
Jul 02 19:57:15.431814 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:15.430965    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:15.431814 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:15.431058    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:15.431814 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:15.431171    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:57:15.431814 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:15.431235    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:16.431217225 +0000 UTC m=+48.578985991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : configmap "kube-rbac-proxy" not found
Jul 02 19:57:15.431814 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:15.431685    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:15.431814 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:15.431737    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:16.4317223 +0000 UTC m=+48.579491063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:16.312828 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:16.312777    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bt4zv_b9f3b082-85fb-4e61-84eb-11697e1dbb76/dns-node-resolver/0.log"
Jul 02 19:57:16.443645 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:16.443604    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:16.443832 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:16.443737    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:16.443908 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:16.443882    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:16.443991 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:16.443960    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:18.443937804 +0000 UTC m=+50.591706584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:16.445180 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:16.445159    2536 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 19:57:16.445322 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:16.445212    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:18.445200367 +0000 UTC m=+50.592969124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : configmap "kube-rbac-proxy" not found
Jul 02 19:57:17.512785 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:17.512760    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xnztd_313b0821-33f6-430d-b1c7-7b1723f5930c/node-ca/0.log"
Jul 02 19:57:18.121974 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:18.121940    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j7xqq_d428d5e0-4fd2-40e1-8cd6-be097512e90c/serve-healthcheck-canary/0.log"
Jul 02 19:57:18.463859 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:18.463819    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:18.464036 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:18.463901    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:18.464036 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:18.464003    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:18.464141 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:18.464085    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:22.464062866 +0000 UTC m=+54.611831624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:18.468971 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:18.468950    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9905aed5-7419-45c0-a28a-aedc1a674c29-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:19.674132 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:19.674089    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:19.674634 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:19.674155    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:19.674634 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:19.674260    2536 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 19:57:19.674634 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:19.674268    2536 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 19:57:19.674634 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:19.674276    2536 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-859cd577c6-cpbqw: secret "image-registry-tls" not found
Jul 02 19:57:19.674634 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:19.674370    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls podName:d625d727-ce77-4086-acd0-b6610616afa8 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:35.674347361 +0000 UTC m=+67.822116124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls") pod "dns-default-hwq8h" (UID: "d625d727-ce77-4086-acd0-b6610616afa8") : secret "dns-default-metrics-tls" not found
Jul 02 19:57:19.674634 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:19.674418    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls podName:50dc66fb-0be2-44a0-93cd-2d9976d4b75f nodeName:}" failed. No retries permitted until 2026-07-02 19:57:35.674400869 +0000 UTC m=+67.822169628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls") pod "image-registry-859cd577c6-cpbqw" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f") : secret "image-registry-tls" not found
Jul 02 19:57:20.618757 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.618725    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5" event={"ID":"d6ca7589-1a14-4546-bf92-872b5ad95e53","Type":"ContainerStarted","Data":"a5457c8c9ebf0f91daa6b2cd80edd0bcd361e6ca12684945b99f9117fbd1f6dd"}
Jul 02 19:57:20.620075 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.620048    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq" event={"ID":"7e344b84-428c-4e6a-9ae9-b69f98cd4bd4","Type":"ContainerStarted","Data":"e2e82288cf3ae9b9fb65bdce4349e5d5d9b01031fd6f7966adfc73abdf6d9225"}
Jul 02 19:57:20.621665 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.621647    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" event={"ID":"48e26d2a-71f9-4547-8172-0073388f3df8","Type":"ContainerStarted","Data":"f1b9103bed8f0f6bfb1f462967d5c261a9e8e293c5e4cf188036a6806d90354d"}
Jul 02 19:57:20.621744 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.621674    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" event={"ID":"48e26d2a-71f9-4547-8172-0073388f3df8","Type":"ContainerStarted","Data":"be5edf65072789a8627c8d5d9566d77264b8bc6cd30032ae9e1af1ab3b21ccbd"}
Jul 02 19:57:20.622518 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.622454    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:20.623761 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.623742    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq"
Jul 02 19:57:20.640355 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.640307    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-787b76ffbd-mm4hq" podStartSLOduration=1.874867633 podStartE2EDuration="8.640274372s" totalImagesPullingTime="6.765406739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:20.63980881 +0000 UTC m=+52.787577588" watchObservedRunningTime="2026-07-02 19:57:20.640274372 +0000 UTC m=+52.788043154"
Jul 02 19:57:20.656920 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:20.656886    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66bc7446c8-d4pf5" podStartSLOduration=1.880244523 podStartE2EDuration="8.656874948s" totalImagesPullingTime="6.776630425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:20.656439907 +0000 UTC m=+52.804208704" watchObservedRunningTime="2026-07-02 19:57:20.656874948 +0000 UTC m=+52.804643768"
Jul 02 19:57:21.626960 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:21.626920    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" event={"ID":"48e26d2a-71f9-4547-8172-0073388f3df8","Type":"ContainerStarted","Data":"21a315d963a9b57aa85a5d91919efab77a851a9ec6868f3aa3e95cc2804b542f"}
Jul 02 19:57:21.650897 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:21.650848    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" podStartSLOduration=2.884389437 podStartE2EDuration="9.650835201s" totalImagesPullingTime="6.766445764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:21.649214207 +0000 UTC m=+53.796982986" watchObservedRunningTime="2026-07-02 19:57:21.650835201 +0000 UTC m=+53.798603980"
Jul 02 19:57:22.498031 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:22.497990    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:22.498205 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:22.498132    2536 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:22.498205 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:22.498194    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls podName:9905aed5-7419-45c0-a28a-aedc1a674c29 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:30.49817979 +0000 UTC m=+62.645948548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zqcmd" (UID: "9905aed5-7419-45c0-a28a-aedc1a674c29") : secret "insights-runtime-extractor-tls" not found
Jul 02 19:57:23.304510 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:23.304470    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:23.304865 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:23.304579    2536 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 02 19:57:23.304865 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:57:23.304639    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert podName:3e2e2791-aba8-4f69-a931-de18d3887569 nodeName:}" failed. No retries permitted until 2026-07-02 19:57:39.304624658 +0000 UTC m=+71.452393415 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-9g79g" (UID: "3e2e2791-aba8-4f69-a931-de18d3887569") : secret "networking-console-plugin-cert" not found
Jul 02 19:57:23.367000 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:23.366960    2536 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" podUID="48e26d2a-71f9-4547-8172-0073388f3df8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 02 19:57:28.544351 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:28.544324    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8dzn4"
Jul 02 19:57:30.556302 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:30.556264    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:30.558703 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:30.558681    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9905aed5-7419-45c0-a28a-aedc1a674c29-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqcmd\" (UID: \"9905aed5-7419-45c0-a28a-aedc1a674c29\") " pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:30.672564 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:30.672531    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-57z4p\""
Jul 02 19:57:30.681445 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:30.681423    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqcmd"
Jul 02 19:57:30.807256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:30.807180    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqcmd"]
Jul 02 19:57:30.810412 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:30.810388    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9905aed5_7419_45c0_a28a_aedc1a674c29.slice/crio-3925342b966560c6bffc43fd5e02c08128a89dc11556e5e3efaaba8f0fdf16db WatchSource:0}: Error finding container 3925342b966560c6bffc43fd5e02c08128a89dc11556e5e3efaaba8f0fdf16db: Status 404 returned error can't find the container with id 3925342b966560c6bffc43fd5e02c08128a89dc11556e5e3efaaba8f0fdf16db
Jul 02 19:57:31.658733 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:31.658695    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqcmd" event={"ID":"9905aed5-7419-45c0-a28a-aedc1a674c29","Type":"ContainerStarted","Data":"3925342b966560c6bffc43fd5e02c08128a89dc11556e5e3efaaba8f0fdf16db"}
Jul 02 19:57:33.366088 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:33.366038    2536 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" podUID="48e26d2a-71f9-4547-8172-0073388f3df8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 02 19:57:34.188380 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.188278    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:57:34.188577 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.188452    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:57:34.192374 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.192350    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 19:57:34.193031 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.193004    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 19:57:34.201010 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.200988    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41653b25-2617-4640-a58f-1853517ff74d-original-pull-secret\") pod \"global-pull-secret-syncer-llksn\" (UID: \"41653b25-2617-4640-a58f-1853517ff74d\") " pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:57:34.201141 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.201125    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6881e29-762a-4296-8c61-1b1774a6577a-metrics-certs\") pod \"network-metrics-daemon-d2pws\" (UID: \"f6881e29-762a-4296-8c61-1b1774a6577a\") " pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:57:34.289273 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.289229    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:34.292458 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.292428    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 19:57:34.303440 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.303405    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 19:57:34.312888 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.312861    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wrbv\" (UniqueName: \"kubernetes.io/projected/cdedba5b-0c4b-453d-b5b9-30da82f6fef3-kube-api-access-4wrbv\") pod \"network-check-target-7mf8w\" (UID: \"cdedba5b-0c4b-453d-b5b9-30da82f6fef3\") " pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:34.477391 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.477373    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tj5bt\""
Jul 02 19:57:34.482489 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.482469    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llksn"
Jul 02 19:57:34.484807 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.484788    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d2pws"
Jul 02 19:57:34.498597 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.498559    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g6jwx\""
Jul 02 19:57:34.502129 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.502110    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:34.646639 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.646609    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llksn"]
Jul 02 19:57:34.650365 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:34.650341    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41653b25_2617_4640_a58f_1853517ff74d.slice/crio-fd446a6e480835f3f569209112bcfbe83519088a63e4dc4fb7f2630266f93d2a WatchSource:0}: Error finding container fd446a6e480835f3f569209112bcfbe83519088a63e4dc4fb7f2630266f93d2a: Status 404 returned error can't find the container with id fd446a6e480835f3f569209112bcfbe83519088a63e4dc4fb7f2630266f93d2a
Jul 02 19:57:34.668743 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.668667    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d2pws"]
Jul 02 19:57:34.671073 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:34.671046    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6881e29_762a_4296_8c61_1b1774a6577a.slice/crio-3fb29b5f4eb0105884964c81124bbdc71dcace089e0ab76316359aab5e520518 WatchSource:0}: Error finding container 3fb29b5f4eb0105884964c81124bbdc71dcace089e0ab76316359aab5e520518: Status 404 returned error can't find the container with id 3fb29b5f4eb0105884964c81124bbdc71dcace089e0ab76316359aab5e520518
Jul 02 19:57:34.671216 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.671186    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqcmd" event={"ID":"9905aed5-7419-45c0-a28a-aedc1a674c29","Type":"ContainerStarted","Data":"0eea17bd809ae858c40d0200268e61163661850d6a963b0f496c29c38be6489c"}
Jul 02 19:57:34.672315 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.672296    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llksn" event={"ID":"41653b25-2617-4640-a58f-1853517ff74d","Type":"ContainerStarted","Data":"fd446a6e480835f3f569209112bcfbe83519088a63e4dc4fb7f2630266f93d2a"}
Jul 02 19:57:34.683038 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:34.683014    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7mf8w"]
Jul 02 19:57:34.686436 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:34.686417    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdedba5b_0c4b_453d_b5b9_30da82f6fef3.slice/crio-1a3da4d65a1e7f23f6bba6838d47341821be99f9136ca133b8c691e8831d7526 WatchSource:0}: Error finding container 1a3da4d65a1e7f23f6bba6838d47341821be99f9136ca133b8c691e8831d7526: Status 404 returned error can't find the container with id 1a3da4d65a1e7f23f6bba6838d47341821be99f9136ca133b8c691e8831d7526
Jul 02 19:57:35.678940 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.678857    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqcmd" event={"ID":"9905aed5-7419-45c0-a28a-aedc1a674c29","Type":"ContainerStarted","Data":"0300e012b7e9bd9018c08845516c8a4fb4c74fd5f6e70a331c679e34ec0864ba"}
Jul 02 19:57:35.680214 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.680179    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7mf8w" event={"ID":"cdedba5b-0c4b-453d-b5b9-30da82f6fef3","Type":"ContainerStarted","Data":"1a3da4d65a1e7f23f6bba6838d47341821be99f9136ca133b8c691e8831d7526"}
Jul 02 19:57:35.682081 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.682051    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d2pws" event={"ID":"f6881e29-762a-4296-8c61-1b1774a6577a","Type":"ContainerStarted","Data":"3fb29b5f4eb0105884964c81124bbdc71dcace089e0ab76316359aab5e520518"}
Jul 02 19:57:35.703327 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.702842    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:35.703327 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.702891    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:35.709861 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.709830    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"image-registry-859cd577c6-cpbqw\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") " pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:35.710239 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.710196    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d625d727-ce77-4086-acd0-b6610616afa8-metrics-tls\") pod \"dns-default-hwq8h\" (UID: \"d625d727-ce77-4086-acd0-b6610616afa8\") " pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:35.877004 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.876978    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w4q8x\""
Jul 02 19:57:35.884707 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.884676    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:35.885127 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.885089    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vrpnh\""
Jul 02 19:57:35.893469 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:35.893224    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:36.071907 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.071807    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hwq8h"]
Jul 02 19:57:36.077608 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:36.077570    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd625d727_ce77_4086_acd0_b6610616afa8.slice/crio-495042e6bbc9e6f2a8f0b90bcf0bc0e607b6fdf8e51695a83523694e67628c78 WatchSource:0}: Error finding container 495042e6bbc9e6f2a8f0b90bcf0bc0e607b6fdf8e51695a83523694e67628c78: Status 404 returned error can't find the container with id 495042e6bbc9e6f2a8f0b90bcf0bc0e607b6fdf8e51695a83523694e67628c78
Jul 02 19:57:36.103636 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.103568    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-859cd577c6-cpbqw"]
Jul 02 19:57:36.108090 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:36.108050    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dc66fb_0be2_44a0_93cd_2d9976d4b75f.slice/crio-ad951765a9b5e68c58edef751007753cb3f4c08bd9699cb3f75a4bbdd534c22b WatchSource:0}: Error finding container ad951765a9b5e68c58edef751007753cb3f4c08bd9699cb3f75a4bbdd534c22b: Status 404 returned error can't find the container with id ad951765a9b5e68c58edef751007753cb3f4c08bd9699cb3f75a4bbdd534c22b
Jul 02 19:57:36.688945 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.688905    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d2pws" event={"ID":"f6881e29-762a-4296-8c61-1b1774a6577a","Type":"ContainerStarted","Data":"d78609642338c5497689868d303cf61ad17ec8610639348e74cd2fa33a262a47"}
Jul 02 19:57:36.689502 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.688976    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d2pws" event={"ID":"f6881e29-762a-4296-8c61-1b1774a6577a","Type":"ContainerStarted","Data":"092286800c9420c81a81ea60e159f3f4d65aed3a40a7004fa67cf6a7569404ac"}
Jul 02 19:57:36.692869 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.692834    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw" event={"ID":"50dc66fb-0be2-44a0-93cd-2d9976d4b75f","Type":"ContainerStarted","Data":"30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39"}
Jul 02 19:57:36.693007 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.692895    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw" event={"ID":"50dc66fb-0be2-44a0-93cd-2d9976d4b75f","Type":"ContainerStarted","Data":"ad951765a9b5e68c58edef751007753cb3f4c08bd9699cb3f75a4bbdd534c22b"}
Jul 02 19:57:36.693496 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.693414    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:57:36.695986 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.695122    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hwq8h" event={"ID":"d625d727-ce77-4086-acd0-b6610616afa8","Type":"ContainerStarted","Data":"495042e6bbc9e6f2a8f0b90bcf0bc0e607b6fdf8e51695a83523694e67628c78"}
Jul 02 19:57:36.718327 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.718253    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d2pws" podStartSLOduration=67.522906369 podStartE2EDuration="1m8.718234415s" totalImagesPullingTime="1.195328046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:36.717006227 +0000 UTC m=+68.864775008" watchObservedRunningTime="2026-07-02 19:57:36.718234415 +0000 UTC m=+68.866003192"
Jul 02 19:57:36.757785 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:36.757738    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw" podStartSLOduration=41.757721265 podStartE2EDuration="41.757721265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:36.756829161 +0000 UTC m=+68.904597943" watchObservedRunningTime="2026-07-02 19:57:36.757721265 +0000 UTC m=+68.905490022"
Jul 02 19:57:39.333836 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.333800    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:39.336649 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.336622    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e2e2791-aba8-4f69-a931-de18d3887569-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-9g79g\" (UID: \"3e2e2791-aba8-4f69-a931-de18d3887569\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:39.493125 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.493090    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-859cd577c6-cpbqw"]
Jul 02 19:57:39.607401 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.607223    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hj56r\""
Jul 02 19:57:39.617588 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.617568    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"
Jul 02 19:57:39.708331 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.707804    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-89d64b9c5-79mjs"]
Jul 02 19:57:39.709871 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.709842    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.817933 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.817898    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-89d64b9c5-79mjs"]
Jul 02 19:57:39.838084 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838057    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a764676c-2904-4b71-b526-a9d6111b1a67-image-registry-private-configuration\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.838256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838097    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a764676c-2904-4b71-b526-a9d6111b1a67-installation-pull-secrets\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.838256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838128    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a764676c-2904-4b71-b526-a9d6111b1a67-registry-certificates\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.838256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838175    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-bound-sa-token\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.838256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838216    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4bcm\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-kube-api-access-k4bcm\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.838256 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838252    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-registry-tls\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.838510 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838342    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a764676c-2904-4b71-b526-a9d6111b1a67-ca-trust-extracted\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.838510 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.838386    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a764676c-2904-4b71-b526-a9d6111b1a67-trusted-ca\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939525 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939493    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a764676c-2904-4b71-b526-a9d6111b1a67-image-registry-private-configuration\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939528    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a764676c-2904-4b71-b526-a9d6111b1a67-installation-pull-secrets\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939560    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a764676c-2904-4b71-b526-a9d6111b1a67-registry-certificates\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939579    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-bound-sa-token\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939606    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4bcm\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-kube-api-access-k4bcm\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939623    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-registry-tls\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939667    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a764676c-2904-4b71-b526-a9d6111b1a67-ca-trust-extracted\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.939715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.939702    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a764676c-2904-4b71-b526-a9d6111b1a67-trusted-ca\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.940972 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.940711    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a764676c-2904-4b71-b526-a9d6111b1a67-registry-certificates\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.940972 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.940847    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a764676c-2904-4b71-b526-a9d6111b1a67-trusted-ca\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.941157 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.941145    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a764676c-2904-4b71-b526-a9d6111b1a67-ca-trust-extracted\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.942607 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.942583    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a764676c-2904-4b71-b526-a9d6111b1a67-image-registry-private-configuration\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.942727 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.942651    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a764676c-2904-4b71-b526-a9d6111b1a67-installation-pull-secrets\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.943218 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.943180    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-registry-tls\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.966154 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.966124    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-bound-sa-token\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:39.968715 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:39.968698    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4bcm\" (UniqueName: \"kubernetes.io/projected/a764676c-2904-4b71-b526-a9d6111b1a67-kube-api-access-k4bcm\") pod \"image-registry-89d64b9c5-79mjs\" (UID: \"a764676c-2904-4b71-b526-a9d6111b1a67\") " pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:40.020451 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:40.020416    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:41.722033 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:41.722004    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7mf8w" event={"ID":"cdedba5b-0c4b-453d-b5b9-30da82f6fef3","Type":"ContainerStarted","Data":"06308f8d4c4164961f6d00db9d015b5ca8c9662c35b22a4557b6277b509868d6"}
Jul 02 19:57:41.722637 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:41.722586    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 19:57:41.760747 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:41.760693    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7mf8w" podStartSLOduration=66.864606795 podStartE2EDuration="1m13.760671513s" totalImagesPullingTime="6.896064718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:41.75963972 +0000 UTC m=+73.907408499" watchObservedRunningTime="2026-07-02 19:57:41.760671513 +0000 UTC m=+73.908440292"
Jul 02 19:57:41.773400 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:41.773331    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-9g79g"]
Jul 02 19:57:41.780413 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:41.780387    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2e2791_aba8_4f69_a931_de18d3887569.slice/crio-931dbfa57255cc74c1ee021bb3323b3f2b84154e6a682ae1550889bd72da100c WatchSource:0}: Error finding container 931dbfa57255cc74c1ee021bb3323b3f2b84154e6a682ae1550889bd72da100c: Status 404 returned error can't find the container with id 931dbfa57255cc74c1ee021bb3323b3f2b84154e6a682ae1550889bd72da100c
Jul 02 19:57:41.816070 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:41.816045    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-89d64b9c5-79mjs"]
Jul 02 19:57:41.828567 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:41.828530    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda764676c_2904_4b71_b526_a9d6111b1a67.slice/crio-cf57be56c63b79141c9c68995626116e7078315384b20d6299be4793deccc0c6 WatchSource:0}: Error finding container cf57be56c63b79141c9c68995626116e7078315384b20d6299be4793deccc0c6: Status 404 returned error can't find the container with id cf57be56c63b79141c9c68995626116e7078315384b20d6299be4793deccc0c6
Jul 02 19:57:42.726952 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.726912    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llksn" event={"ID":"41653b25-2617-4640-a58f-1853517ff74d","Type":"ContainerStarted","Data":"3f3f279c05435e2ea68ec50c27e0cf129b84af765f719fb347a6c94d3ee7a86c"}
Jul 02 19:57:42.728097 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.728074    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g" event={"ID":"3e2e2791-aba8-4f69-a931-de18d3887569","Type":"ContainerStarted","Data":"931dbfa57255cc74c1ee021bb3323b3f2b84154e6a682ae1550889bd72da100c"}
Jul 02 19:57:42.729674 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.729643    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs" event={"ID":"a764676c-2904-4b71-b526-a9d6111b1a67","Type":"ContainerStarted","Data":"94ce7cf13f7181da3021eded3f2a7b4f8ced01cc5cf4b2783d2624d902d9822d"}
Jul 02 19:57:42.729798 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.729686    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs" event={"ID":"a764676c-2904-4b71-b526-a9d6111b1a67","Type":"ContainerStarted","Data":"cf57be56c63b79141c9c68995626116e7078315384b20d6299be4793deccc0c6"}
Jul 02 19:57:42.729798 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.729748    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:57:42.731320 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.731296    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hwq8h" event={"ID":"d625d727-ce77-4086-acd0-b6610616afa8","Type":"ContainerStarted","Data":"b816fbec42e28110502a8a1a934f33f697f8c7478774085ed98e6296e5d69458"}
Jul 02 19:57:42.731423 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.731340    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hwq8h" event={"ID":"d625d727-ce77-4086-acd0-b6610616afa8","Type":"ContainerStarted","Data":"ca683c1f1aec01fafe2ea2e41c31fd06d644b3fefa04042829c88adbcd28dfb5"}
Jul 02 19:57:42.731423 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.731369    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:42.733232 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.733209    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqcmd" event={"ID":"9905aed5-7419-45c0-a28a-aedc1a674c29","Type":"ContainerStarted","Data":"db8ec49125b58441605c87c24e75125f901412e6233a518e35dd5d44fe0886f6"}
Jul 02 19:57:42.846986 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.846926    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-llksn" podStartSLOduration=67.914334273 podStartE2EDuration="1m14.846910106s" totalImagesPullingTime="6.932575833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:42.783046614 +0000 UTC m=+74.930815392" watchObservedRunningTime="2026-07-02 19:57:42.846910106 +0000 UTC m=+74.994678885"
Jul 02 19:57:42.847239 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.847209    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs" podStartSLOduration=3.847202424 podStartE2EDuration="3.847202424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:42.845589477 +0000 UTC m=+74.993358257" watchObservedRunningTime="2026-07-02 19:57:42.847202424 +0000 UTC m=+74.994971206"
Jul 02 19:57:42.880774 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:42.880714    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hwq8h" podStartSLOduration=34.374506693 podStartE2EDuration="39.880693539s" totalImagesPullingTime="5.506186846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:57:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:42.879956185 +0000 UTC m=+75.027724964" watchObservedRunningTime="2026-07-02 19:57:42.880693539 +0000 UTC m=+75.028462319"
Jul 02 19:57:43.365641 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.365603    2536 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" podUID="48e26d2a-71f9-4547-8172-0073388f3df8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 02 19:57:43.365775 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.365666    2536 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m"
Jul 02 19:57:43.366103 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.366075    2536 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"21a315d963a9b57aa85a5d91919efab77a851a9ec6868f3aa3e95cc2804b542f"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 02 19:57:43.366140 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.366121    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" podUID="48e26d2a-71f9-4547-8172-0073388f3df8" containerName="service-proxy" containerID="cri-o://21a315d963a9b57aa85a5d91919efab77a851a9ec6868f3aa3e95cc2804b542f" gracePeriod=30
Jul 02 19:57:43.738382 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.738350    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g" event={"ID":"3e2e2791-aba8-4f69-a931-de18d3887569","Type":"ContainerStarted","Data":"982e9f66a9510fed61034bdeb94fa269cffde303fc1d389145ceb28562cf79ff"}
Jul 02 19:57:43.740777 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.740756    2536 generic.go:356] "Generic (PLEG): container finished" podID="48e26d2a-71f9-4547-8172-0073388f3df8" containerID="21a315d963a9b57aa85a5d91919efab77a851a9ec6868f3aa3e95cc2804b542f" exitCode=2
Jul 02 19:57:43.740864 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.740842    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" event={"ID":"48e26d2a-71f9-4547-8172-0073388f3df8","Type":"ContainerDied","Data":"21a315d963a9b57aa85a5d91919efab77a851a9ec6868f3aa3e95cc2804b542f"}
Jul 02 19:57:43.740902 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.740878    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f5c9b87db-pg47m" event={"ID":"48e26d2a-71f9-4547-8172-0073388f3df8","Type":"ContainerStarted","Data":"f8244d6c382d69ba7eb279717356295361797ad430fd87394fc962353e45a1f1"}
Jul 02 19:57:43.778979 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.778901    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zqcmd" podStartSLOduration=19.152471574 podStartE2EDuration="29.778888793s" totalImagesPullingTime="10.626417219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:57:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:42.932956529 +0000 UTC m=+75.080725307" watchObservedRunningTime="2026-07-02 19:57:43.778888793 +0000 UTC m=+75.926657572"
Jul 02 19:57:43.779458 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:43.779422    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-9g79g" podStartSLOduration=35.31705748 podStartE2EDuration="36.779409838s" totalImagesPullingTime="1.462352358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:43.778781798 +0000 UTC m=+75.926550577" watchObservedRunningTime="2026-07-02 19:57:43.779409838 +0000 UTC m=+75.927178619"
Jul 02 19:57:52.743158 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:52.743053    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hwq8h"
Jul 02 19:57:54.275818 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.275782    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l254d"]
Jul 02 19:57:54.325326 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.325275    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.332377 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.332354    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qvx5c\""
Jul 02 19:57:54.332377 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.332358    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 19:57:54.332572 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.332390    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 19:57:54.332572 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.332423    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 19:57:54.332724 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.332706    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 19:57:54.332724 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.332721    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 19:57:54.332855 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.332753    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 19:57:54.439155 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439123    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-textfile\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439155 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439157    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-256l6\" (UniqueName: \"kubernetes.io/projected/e0fef006-37fc-4281-9bb8-d174c74754cd-kube-api-access-256l6\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439373 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439184    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439373 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439243    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-wtmp\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439373 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439301    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-tls\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439373 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439339    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-root\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439373 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439364    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439373 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439382    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-sys\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.439578 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.439402    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0fef006-37fc-4281-9bb8-d174c74754cd-metrics-client-ca\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540366    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540398    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-sys\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540435 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540419    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0fef006-37fc-4281-9bb8-d174c74754cd-metrics-client-ca\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540440    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-textfile\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540469    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-256l6\" (UniqueName: \"kubernetes.io/projected/e0fef006-37fc-4281-9bb8-d174c74754cd-kube-api-access-256l6\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540492    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-sys\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540514    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540546    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-wtmp\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540570    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-tls\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540597    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-root\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.540659 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540656    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-root\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.541075 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540736    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-wtmp\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.541075 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.540850    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-textfile\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.541075 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.541055    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0fef006-37fc-4281-9bb8-d174c74754cd-metrics-client-ca\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.541415 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.541157    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.542856 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.542836    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.543055 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.543026    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e0fef006-37fc-4281-9bb8-d174c74754cd-node-exporter-tls\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.556915 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.556891    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-256l6\" (UniqueName: \"kubernetes.io/projected/e0fef006-37fc-4281-9bb8-d174c74754cd-kube-api-access-256l6\") pod \"node-exporter-l254d\" (UID: \"e0fef006-37fc-4281-9bb8-d174c74754cd\") " pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.634767 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.634734    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l254d"
Jul 02 19:57:54.645061 ip-10-0-142-101 kubenswrapper[2536]: W0702 19:57:54.645037    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0fef006_37fc_4281_9bb8_d174c74754cd.slice/crio-9fa0fe1b451b4e51c1066e8eda72b07743e17414dba3880d9cf539c4b1892427 WatchSource:0}: Error finding container 9fa0fe1b451b4e51c1066e8eda72b07743e17414dba3880d9cf539c4b1892427: Status 404 returned error can't find the container with id 9fa0fe1b451b4e51c1066e8eda72b07743e17414dba3880d9cf539c4b1892427
Jul 02 19:57:54.783007 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:54.782977    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l254d" event={"ID":"e0fef006-37fc-4281-9bb8-d174c74754cd","Type":"ContainerStarted","Data":"9fa0fe1b451b4e51c1066e8eda72b07743e17414dba3880d9cf539c4b1892427"}
Jul 02 19:57:56.791633 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:56.791593    2536 generic.go:356] "Generic (PLEG): container finished" podID="e0fef006-37fc-4281-9bb8-d174c74754cd" containerID="13b43fd7ab78d8280e2167ae8bb4432265b1bcc274cc9fafe3573c9fab0ab11f" exitCode=0
Jul 02 19:57:56.791994 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:56.791680    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l254d" event={"ID":"e0fef006-37fc-4281-9bb8-d174c74754cd","Type":"ContainerDied","Data":"13b43fd7ab78d8280e2167ae8bb4432265b1bcc274cc9fafe3573c9fab0ab11f"}
Jul 02 19:57:57.797240 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:57.797205    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l254d" event={"ID":"e0fef006-37fc-4281-9bb8-d174c74754cd","Type":"ContainerStarted","Data":"39642ddcd1c34f11addae1784e93ce70da06e6cb973f5cf851967ffc8c7f70d1"}
Jul 02 19:57:57.797622 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:57.797328    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l254d" event={"ID":"e0fef006-37fc-4281-9bb8-d174c74754cd","Type":"ContainerStarted","Data":"3d9744df6d742a61116dbf144b9fb4c91571e453cce341d02af1d063cc909ddb"}
Jul 02 19:57:57.819050 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:57.819002    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l254d" podStartSLOduration=2.425516006 podStartE2EDuration="3.81898813s" totalImagesPullingTime="1.324252627s" totalInitContainerRuntime="69.219497ms" isStatefulPod=true podCreationTimestamp="2026-07-02 19:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:57:57.81752574 +0000 UTC m=+89.965294533" watchObservedRunningTime="2026-07-02 19:57:57.81898813 +0000 UTC m=+89.966756908"
Jul 02 19:57:59.714344 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:57:59.714315    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:58:00.025082 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:00.025001    2536 patch_prober.go:28] interesting pod/image-registry-89d64b9c5-79mjs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 19:58:00.025082 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:00.025054    2536 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs" podUID="a764676c-2904-4b71-b526-a9d6111b1a67" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 19:58:03.746012 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:03.745984    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-89d64b9c5-79mjs"
Jul 02 19:58:04.730038 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:04.729977    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw" podUID="50dc66fb-0be2-44a0-93cd-2d9976d4b75f" containerName="registry" containerID="cri-o://30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39" gracePeriod=30
Jul 02 19:58:04.967812 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:04.967789    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:58:05.117586 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117492    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-installation-pull-secrets\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.117586 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117538    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-certificates\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.117586 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117588    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-ca-trust-extracted\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.117856 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117627    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-image-registry-private-configuration\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.117856 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117664    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-bound-sa-token\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.117856 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117694    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.117856 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117723    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxr25\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-kube-api-access-fxr25\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.117856 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117754    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-trusted-ca\") pod \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\" (UID: \"50dc66fb-0be2-44a0-93cd-2d9976d4b75f\") "
Jul 02 19:58:05.118088 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.117979    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:58:05.118325 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.118301    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 19:58:05.120235 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.120198    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:58:05.120477 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.120449    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-kube-api-access-fxr25" (OuterVolumeSpecName: "kube-api-access-fxr25") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "kube-api-access-fxr25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:58:05.120567 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.120522    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:58:05.120567 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.120521    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 19:58:05.120699 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.120652    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 19:58:05.126523 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.126500    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "50dc66fb-0be2-44a0-93cd-2d9976d4b75f" (UID: "50dc66fb-0be2-44a0-93cd-2d9976d4b75f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 19:58:05.218810 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218778    2536 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.218810 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218805    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxr25\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-kube-api-access-fxr25\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.218810 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218815    2536 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-trusted-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.219016 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218823    2536 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-installation-pull-secrets\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.219016 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218832    2536 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-registry-certificates\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.219016 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218841    2536 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-ca-trust-extracted\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.219016 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218850    2536 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-image-registry-private-configuration\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.219016 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.218859    2536 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50dc66fb-0be2-44a0-93cd-2d9976d4b75f-bound-sa-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 19:58:05.832958 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.832925    2536 generic.go:356] "Generic (PLEG): container finished" podID="50dc66fb-0be2-44a0-93cd-2d9976d4b75f" containerID="30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39" exitCode=0
Jul 02 19:58:05.833116 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.832967    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw" event={"ID":"50dc66fb-0be2-44a0-93cd-2d9976d4b75f","Type":"ContainerDied","Data":"30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39"}
Jul 02 19:58:05.833116 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.832983    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw"
Jul 02 19:58:05.833116 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.833028    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-859cd577c6-cpbqw" event={"ID":"50dc66fb-0be2-44a0-93cd-2d9976d4b75f","Type":"ContainerDied","Data":"ad951765a9b5e68c58edef751007753cb3f4c08bd9699cb3f75a4bbdd534c22b"}
Jul 02 19:58:05.833116 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.833048    2536 scope.go:117] "RemoveContainer" containerID="30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39"
Jul 02 19:58:05.841611 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.841594    2536 scope.go:117] "RemoveContainer" containerID="30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39"
Jul 02 19:58:05.841909 ip-10-0-142-101 kubenswrapper[2536]: E0702 19:58:05.841886    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39\": container with ID starting with 30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39 not found: ID does not exist" containerID="30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39"
Jul 02 19:58:05.841998 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.841920    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39"} err="failed to get container status \"30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39\": rpc error: code = NotFound desc = could not find container \"30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39\": container with ID starting with 30d1720ad37e7f686f0eb26c7ab373d70efa7e3d348be4ea9cd5abda0a1fac39 not found: ID does not exist"
Jul 02 19:58:05.853745 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.853715    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-859cd577c6-cpbqw"]
Jul 02 19:58:05.859469 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:05.859451    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-859cd577c6-cpbqw"]
Jul 02 19:58:06.367606 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:06.367570    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50dc66fb-0be2-44a0-93cd-2d9976d4b75f" path="/var/lib/kubelet/pods/50dc66fb-0be2-44a0-93cd-2d9976d4b75f/volumes"
Jul 02 19:58:12.735648 ip-10-0-142-101 kubenswrapper[2536]: I0702 19:58:12.735614    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7mf8w"
Jul 02 20:01:28.274646 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:01:28.274619    2536 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 20:02:25.285917 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.285840    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"]
Jul 02 20:02:25.286379 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.286062    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dc66fb-0be2-44a0-93cd-2d9976d4b75f" containerName="registry"
Jul 02 20:02:25.286379 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.286071    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dc66fb-0be2-44a0-93cd-2d9976d4b75f" containerName="registry"
Jul 02 20:02:25.286379 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.286118    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="50dc66fb-0be2-44a0-93cd-2d9976d4b75f" containerName="registry"
Jul 02 20:02:25.288995 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.288979    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.292516 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.292495    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 02 20:02:25.292642 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.292573    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 02 20:02:25.292642 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.292597    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mh7lb\""
Jul 02 20:02:25.292642 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.292636    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 02 20:02:25.293204 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.293182    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 02 20:02:25.309361 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.309338    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-webhook-cert\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.309494 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.309367    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-manifests\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.309494 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.309410    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw2p8\" (UniqueName: \"kubernetes.io/projected/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-kube-api-access-hw2p8\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.309494 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.309427    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.309494 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.309474    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-tmp\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.319184 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.319164    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"]
Jul 02 20:02:25.409902 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.409869    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw2p8\" (UniqueName: \"kubernetes.io/projected/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-kube-api-access-hw2p8\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.410082 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.409918    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.410082 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.409945    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-tmp\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.410082 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.409982    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-webhook-cert\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.410082 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.410009    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-manifests\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.410557 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.410526    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-manifests\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.412503 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.412482    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-tmp\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.412733 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.412713    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.412777 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.412713    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-webhook-cert\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.421721 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.421697    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw2p8\" (UniqueName: \"kubernetes.io/projected/2bd5f4a8-30c9-44f7-bdcd-09d68df95759-kube-api-access-hw2p8\") pod \"opendatahub-operator-controller-manager-66c8797c58-9w5cd\" (UID: \"2bd5f4a8-30c9-44f7-bdcd-09d68df95759\") " pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.598985 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.598897    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:25.737069 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.737046    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"]
Jul 02 20:02:25.739350 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:02:25.739323    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd5f4a8_30c9_44f7_bdcd_09d68df95759.slice/crio-9d6b127920b0577e1f85dead6ebe1a230ec39e5f90005fad1a24716652be49a9 WatchSource:0}: Error finding container 9d6b127920b0577e1f85dead6ebe1a230ec39e5f90005fad1a24716652be49a9: Status 404 returned error can't find the container with id 9d6b127920b0577e1f85dead6ebe1a230ec39e5f90005fad1a24716652be49a9
Jul 02 20:02:25.741037 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:25.741019    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 20:02:26.692301 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:26.692213    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd" event={"ID":"2bd5f4a8-30c9-44f7-bdcd-09d68df95759","Type":"ContainerStarted","Data":"9d6b127920b0577e1f85dead6ebe1a230ec39e5f90005fad1a24716652be49a9"}
Jul 02 20:02:29.704617 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:29.704585    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd" event={"ID":"2bd5f4a8-30c9-44f7-bdcd-09d68df95759","Type":"ContainerStarted","Data":"4b3cbb07732a4177728f3da8531c356e409f5fc29a147e94fba8649ada160fb1"}
Jul 02 20:02:30.709250 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:30.709219    2536 generic.go:356] "Generic (PLEG): container finished" podID="2bd5f4a8-30c9-44f7-bdcd-09d68df95759" containerID="4b3cbb07732a4177728f3da8531c356e409f5fc29a147e94fba8649ada160fb1" exitCode=0
Jul 02 20:02:30.709711 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:30.709264    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd" event={"ID":"2bd5f4a8-30c9-44f7-bdcd-09d68df95759","Type":"ContainerDied","Data":"4b3cbb07732a4177728f3da8531c356e409f5fc29a147e94fba8649ada160fb1"}
Jul 02 20:02:31.714242 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:31.714209    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd" event={"ID":"2bd5f4a8-30c9-44f7-bdcd-09d68df95759","Type":"ContainerStarted","Data":"e83658d1043dfd30215b5c0ddeab9fc5df370c483e3ed901a3eb322e494329cb"}
Jul 02 20:02:31.714645 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:31.714330    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:31.748032 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:31.747982    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd" podStartSLOduration=2.394751048 podStartE2EDuration="6.747966845s" totalImagesPullingTime="3.273220574s" totalInitContainerRuntime="1.079995223s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:02:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:02:31.746715581 +0000 UTC m=+363.894484359" watchObservedRunningTime="2026-07-02 20:02:31.747966845 +0000 UTC m=+363.895735623"
Jul 02 20:02:42.720221 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:42.720189    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66c8797c58-9w5cd"
Jul 02 20:02:45.171176 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.171144    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"]
Jul 02 20:02:45.184183 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.184163    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.190680 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.190656    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 02 20:02:45.190765 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.190665    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 02 20:02:45.190765 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.190739    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 02 20:02:45.191545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.191532    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 02 20:02:45.197427 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.197408    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"]
Jul 02 20:02:45.250877 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.250849    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28464414-9c2c-493d-9b4f-0c993fa4afa6-tls-certs\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.251052 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.250900    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28464414-9c2c-493d-9b4f-0c993fa4afa6-tmp\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.251052 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.250926    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxcbm\" (UniqueName: \"kubernetes.io/projected/28464414-9c2c-493d-9b4f-0c993fa4afa6-kube-api-access-jxcbm\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.351600 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.351564    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28464414-9c2c-493d-9b4f-0c993fa4afa6-tmp\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.351779 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.351611    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxcbm\" (UniqueName: \"kubernetes.io/projected/28464414-9c2c-493d-9b4f-0c993fa4afa6-kube-api-access-jxcbm\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.351779 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.351647    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28464414-9c2c-493d-9b4f-0c993fa4afa6-tls-certs\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.351779 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:45.351760    2536 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 02 20:02:45.351943 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:45.351837    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28464414-9c2c-493d-9b4f-0c993fa4afa6-tls-certs podName:28464414-9c2c-493d-9b4f-0c993fa4afa6 nodeName:}" failed. No retries permitted until 2026-07-02 20:02:45.851817894 +0000 UTC m=+377.999586651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/28464414-9c2c-493d-9b4f-0c993fa4afa6-tls-certs") pod "kube-auth-proxy-5b7dc9b6d5-cxmst" (UID: "28464414-9c2c-493d-9b4f-0c993fa4afa6") : secret "kube-auth-proxy-tls" not found
Jul 02 20:02:45.353871 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.353853    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28464414-9c2c-493d-9b4f-0c993fa4afa6-tmp\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.384299 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.384265    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxcbm\" (UniqueName: \"kubernetes.io/projected/28464414-9c2c-493d-9b4f-0c993fa4afa6-kube-api-access-jxcbm\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.432006 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.431920    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hxpt9"]
Jul 02 20:02:45.435236 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.435220    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:45.437878 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.437858    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 02 20:02:45.438085 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.438072    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gmmr2\""
Jul 02 20:02:45.461174 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.461148    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hxpt9"]
Jul 02 20:02:45.553588 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.553548    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnkvr\" (UniqueName: \"kubernetes.io/projected/8a605196-2ef8-46cb-9d32-5f634f27e4cb-kube-api-access-wnkvr\") pod \"odh-model-controller-7998c67dc4-hxpt9\" (UID: \"8a605196-2ef8-46cb-9d32-5f634f27e4cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:45.553764 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.553600    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a605196-2ef8-46cb-9d32-5f634f27e4cb-cert\") pod \"odh-model-controller-7998c67dc4-hxpt9\" (UID: \"8a605196-2ef8-46cb-9d32-5f634f27e4cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:45.654148 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.654113    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a605196-2ef8-46cb-9d32-5f634f27e4cb-cert\") pod \"odh-model-controller-7998c67dc4-hxpt9\" (UID: \"8a605196-2ef8-46cb-9d32-5f634f27e4cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:45.654354 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.654218    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnkvr\" (UniqueName: \"kubernetes.io/projected/8a605196-2ef8-46cb-9d32-5f634f27e4cb-kube-api-access-wnkvr\") pod \"odh-model-controller-7998c67dc4-hxpt9\" (UID: \"8a605196-2ef8-46cb-9d32-5f634f27e4cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:45.654354 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:45.654259    2536 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 02 20:02:45.654354 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:45.654331    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a605196-2ef8-46cb-9d32-5f634f27e4cb-cert podName:8a605196-2ef8-46cb-9d32-5f634f27e4cb nodeName:}" failed. No retries permitted until 2026-07-02 20:02:46.154315002 +0000 UTC m=+378.302083761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8a605196-2ef8-46cb-9d32-5f634f27e4cb-cert") pod "odh-model-controller-7998c67dc4-hxpt9" (UID: "8a605196-2ef8-46cb-9d32-5f634f27e4cb") : secret "odh-model-controller-webhook-cert" not found
Jul 02 20:02:45.675552 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.675522    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnkvr\" (UniqueName: \"kubernetes.io/projected/8a605196-2ef8-46cb-9d32-5f634f27e4cb-kube-api-access-wnkvr\") pod \"odh-model-controller-7998c67dc4-hxpt9\" (UID: \"8a605196-2ef8-46cb-9d32-5f634f27e4cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:45.856351 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.856265    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28464414-9c2c-493d-9b4f-0c993fa4afa6-tls-certs\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:45.858719 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:45.858701    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28464414-9c2c-493d-9b4f-0c993fa4afa6-tls-certs\") pod \"kube-auth-proxy-5b7dc9b6d5-cxmst\" (UID: \"28464414-9c2c-493d-9b4f-0c993fa4afa6\") " pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:46.092844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.092810    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"
Jul 02 20:02:46.159041 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.159010    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a605196-2ef8-46cb-9d32-5f634f27e4cb-cert\") pod \"odh-model-controller-7998c67dc4-hxpt9\" (UID: \"8a605196-2ef8-46cb-9d32-5f634f27e4cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:46.161356 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.161334    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a605196-2ef8-46cb-9d32-5f634f27e4cb-cert\") pod \"odh-model-controller-7998c67dc4-hxpt9\" (UID: \"8a605196-2ef8-46cb-9d32-5f634f27e4cb\") " pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:46.227419 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.227384    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst"]
Jul 02 20:02:46.231180 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:02:46.231153    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28464414_9c2c_493d_9b4f_0c993fa4afa6.slice/crio-4184c72643128528b1b2f665e3cc330687c803d5ae9965197a3bbaaa75351f3a WatchSource:0}: Error finding container 4184c72643128528b1b2f665e3cc330687c803d5ae9965197a3bbaaa75351f3a: Status 404 returned error can't find the container with id 4184c72643128528b1b2f665e3cc330687c803d5ae9965197a3bbaaa75351f3a
Jul 02 20:02:46.344804 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.344774    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:46.490651 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.490619    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hxpt9"]
Jul 02 20:02:46.493828 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:02:46.493801    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a605196_2ef8_46cb_9d32_5f634f27e4cb.slice/crio-6c9291f5dd4d834bba1c1530c4a74bcb366bc133dc3727c5a2a3b400dfd8a0e5 WatchSource:0}: Error finding container 6c9291f5dd4d834bba1c1530c4a74bcb366bc133dc3727c5a2a3b400dfd8a0e5: Status 404 returned error can't find the container with id 6c9291f5dd4d834bba1c1530c4a74bcb366bc133dc3727c5a2a3b400dfd8a0e5
Jul 02 20:02:46.557556 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.557528    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jffl"]
Jul 02 20:02:46.561823 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.561805    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:46.566232 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.566216    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 02 20:02:46.661974 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.661877    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njjdd\" (UniqueName: \"kubernetes.io/projected/ff5d9add-73f8-4038-aaa2-672dadeae79d-kube-api-access-njjdd\") pod \"model-serving-api-54d89bd5fc-9jffl\" (UID: \"ff5d9add-73f8-4038-aaa2-672dadeae79d\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:46.661974 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.661912    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5d9add-73f8-4038-aaa2-672dadeae79d-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jffl\" (UID: \"ff5d9add-73f8-4038-aaa2-672dadeae79d\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:46.697696 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.697236    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jffl"]
Jul 02 20:02:46.762874 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.762849    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njjdd\" (UniqueName: \"kubernetes.io/projected/ff5d9add-73f8-4038-aaa2-672dadeae79d-kube-api-access-njjdd\") pod \"model-serving-api-54d89bd5fc-9jffl\" (UID: \"ff5d9add-73f8-4038-aaa2-672dadeae79d\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:46.763118 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.762882    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5d9add-73f8-4038-aaa2-672dadeae79d-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jffl\" (UID: \"ff5d9add-73f8-4038-aaa2-672dadeae79d\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:46.765430 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.765406    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff5d9add-73f8-4038-aaa2-672dadeae79d-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jffl\" (UID: \"ff5d9add-73f8-4038-aaa2-672dadeae79d\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:46.771350 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.771316    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9" event={"ID":"8a605196-2ef8-46cb-9d32-5f634f27e4cb","Type":"ContainerStarted","Data":"6c9291f5dd4d834bba1c1530c4a74bcb366bc133dc3727c5a2a3b400dfd8a0e5"}
Jul 02 20:02:46.772458 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.772435    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst" event={"ID":"28464414-9c2c-493d-9b4f-0c993fa4afa6","Type":"ContainerStarted","Data":"4184c72643128528b1b2f665e3cc330687c803d5ae9965197a3bbaaa75351f3a"}
Jul 02 20:02:46.779133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.779109    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njjdd\" (UniqueName: \"kubernetes.io/projected/ff5d9add-73f8-4038-aaa2-672dadeae79d-kube-api-access-njjdd\") pod \"model-serving-api-54d89bd5fc-9jffl\" (UID: \"ff5d9add-73f8-4038-aaa2-672dadeae79d\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:46.872082 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:46.872044    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:47.003812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.003781    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jffl"]
Jul 02 20:02:47.006613 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:02:47.006587    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5d9add_73f8_4038_aaa2_672dadeae79d.slice/crio-3ed36aaf35382314a0950d8433a61e25b79b4496cbcb80d4c7266a0dd8628daa WatchSource:0}: Error finding container 3ed36aaf35382314a0950d8433a61e25b79b4496cbcb80d4c7266a0dd8628daa: Status 404 returned error can't find the container with id 3ed36aaf35382314a0950d8433a61e25b79b4496cbcb80d4c7266a0dd8628daa
Jul 02 20:02:47.272135 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.272044    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"]
Jul 02 20:02:47.275771 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.275746    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.284233 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.284197    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 02 20:02:47.284233 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.284211    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 02 20:02:47.284441 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.284338    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2d96n\""
Jul 02 20:02:47.284710 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.284548    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 02 20:02:47.284710 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.284708    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 02 20:02:47.298168 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.298143    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"]
Jul 02 20:02:47.368490 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.368459    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/942b340c-b31e-4c63-9acc-466cef257a10-local-certs\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.368644 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.368509    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-cacerts\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.368644 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.368546    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.368644 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.368597    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5w92\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-kube-api-access-m5w92\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.368644 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.368625    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.368881 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.368655    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-istio-token\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.368881 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.368689    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.469863 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.469785    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-cacerts\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.469863 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.469837    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.470105 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.469891    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5w92\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-kube-api-access-m5w92\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.470105 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.469919    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.470105 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.469947    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-istio-token\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.470105 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.469975    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.470105 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.470016    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/942b340c-b31e-4c63-9acc-466cef257a10-local-certs\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.471857 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.470849    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.473017 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.472969    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-cacerts\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.474681 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.474647    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/942b340c-b31e-4c63-9acc-466cef257a10-local-certs\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.474681 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.474675    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.475199 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.475178    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.482057 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.482002    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5w92\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-kube-api-access-m5w92\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.482781 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.482740    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-istio-token\") pod \"istiod-openshift-gateway-7574459656-55q4s\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.589150 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.589065    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:02:47.772408 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.772376    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"]
Jul 02 20:02:47.773822 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:02:47.773796    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod942b340c_b31e_4c63_9acc_466cef257a10.slice/crio-4b2c2870f6362d231619bae33e2425964fc7d3720091a0ea9f371e24fee9a08f WatchSource:0}: Error finding container 4b2c2870f6362d231619bae33e2425964fc7d3720091a0ea9f371e24fee9a08f: Status 404 returned error can't find the container with id 4b2c2870f6362d231619bae33e2425964fc7d3720091a0ea9f371e24fee9a08f
Jul 02 20:02:47.779317 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:47.779229    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9jffl" event={"ID":"ff5d9add-73f8-4038-aaa2-672dadeae79d","Type":"ContainerStarted","Data":"3ed36aaf35382314a0950d8433a61e25b79b4496cbcb80d4c7266a0dd8628daa"}
Jul 02 20:02:48.787538 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:48.787501    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s" event={"ID":"942b340c-b31e-4c63-9acc-466cef257a10","Type":"ContainerStarted","Data":"4b2c2870f6362d231619bae33e2425964fc7d3720091a0ea9f371e24fee9a08f"}
Jul 02 20:02:50.797993 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.797957    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst" event={"ID":"28464414-9c2c-493d-9b4f-0c993fa4afa6","Type":"ContainerStarted","Data":"b85657377cd02b06f888994bf62e7bb797a823c4db294b5f635bdcf22710ed99"}
Jul 02 20:02:50.799560 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.799536    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9jffl" event={"ID":"ff5d9add-73f8-4038-aaa2-672dadeae79d","Type":"ContainerStarted","Data":"838a8043d940a0e2eff536e501ed985ea654d6aa3775248b9d4909e96264e017"}
Jul 02 20:02:50.799734 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.799585    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:02:50.800975 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.800950    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9" event={"ID":"8a605196-2ef8-46cb-9d32-5f634f27e4cb","Type":"ContainerStarted","Data":"5c99ae9136b444c4d5e8f9b0438e0269ba21134a29c9b3ac49ca5a771359df22"}
Jul 02 20:02:50.801110 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.801007    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:02:50.817411 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.817355    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b7dc9b6d5-cxmst" podStartSLOduration=1.642115902 podStartE2EDuration="5.817339658s" totalImagesPullingTime="4.175223756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:02:50.816095767 +0000 UTC m=+382.963864559" watchObservedRunningTime="2026-07-02 20:02:50.817339658 +0000 UTC m=+382.965108437"
Jul 02 20:02:50.838177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.838118    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9" podStartSLOduration=1.933881511 podStartE2EDuration="5.838102044s" totalImagesPullingTime="3.904220533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:02:50.836713105 +0000 UTC m=+382.984481885" watchObservedRunningTime="2026-07-02 20:02:50.838102044 +0000 UTC m=+382.985870824"
Jul 02 20:02:50.857823 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:50.857760    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9jffl" podStartSLOduration=1.466574316 podStartE2EDuration="4.857742008s" totalImagesPullingTime="3.391167692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:02:50.856500655 +0000 UTC m=+383.004269425" watchObservedRunningTime="2026-07-02 20:02:50.857742008 +0000 UTC m=+383.005510788"
Jul 02 20:02:56.552793 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.552765    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"]
Jul 02 20:02:56.556146 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.556131    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:56.560219 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.560199    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sbs5d\""
Jul 02 20:02:56.560532 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.560516    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 02 20:02:56.573160 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.573137    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"]
Jul 02 20:02:56.594108 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.594080    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"]
Jul 02 20:02:56.597047 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.597034    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:56.601181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.601155    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 02 20:02:56.601311 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.601159    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wdmtf\""
Jul 02 20:02:56.611128 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.611106    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"]
Jul 02 20:02:56.643029 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.643006    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:56.643029 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.643031    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5zts\" (UID: \"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:56.643159 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.643069    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-495jt\" (UniqueName: \"kubernetes.io/projected/ce4c56eb-3090-42bc-b0ce-a3809301e56b-kube-api-access-495jt\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:56.643159 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.643104    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82n2g\" (UniqueName: \"kubernetes.io/projected/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-kube-api-access-82n2g\") pod \"kserve-controller-manager-d6f4b65b8-c5zts\" (UID: \"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:56.744311 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.744269    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:56.744311 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.744317    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5zts\" (UID: \"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:56.744529 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.744351    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-495jt\" (UniqueName: \"kubernetes.io/projected/ce4c56eb-3090-42bc-b0ce-a3809301e56b-kube-api-access-495jt\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:56.744529 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.744380    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82n2g\" (UniqueName: \"kubernetes.io/projected/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-kube-api-access-82n2g\") pod \"kserve-controller-manager-d6f4b65b8-c5zts\" (UID: \"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:56.744529 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:56.744424    2536 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 02 20:02:56.744529 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:56.744461    2536 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 02 20:02:56.744529 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:56.744485    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert podName:ce4c56eb-3090-42bc-b0ce-a3809301e56b nodeName:}" failed. No retries permitted until 2026-07-02 20:02:57.244468724 +0000 UTC m=+389.392237482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert") pod "llmisvc-controller-manager-777c6bd7d5-rszpf" (UID: "ce4c56eb-3090-42bc-b0ce-a3809301e56b") : secret "llmisvc-webhook-server-cert" not found
Jul 02 20:02:56.744723 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:56.744552    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-cert podName:8d39fc83-8f0b-42e4-b01f-19cb107c4a4b nodeName:}" failed. No retries permitted until 2026-07-02 20:02:57.244534546 +0000 UTC m=+389.392303304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-cert") pod "kserve-controller-manager-d6f4b65b8-c5zts" (UID: "8d39fc83-8f0b-42e4-b01f-19cb107c4a4b") : secret "kserve-webhook-server-cert" not found
Jul 02 20:02:56.772842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.772815    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82n2g\" (UniqueName: \"kubernetes.io/projected/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-kube-api-access-82n2g\") pod \"kserve-controller-manager-d6f4b65b8-c5zts\" (UID: \"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:56.773212 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:56.773196    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-495jt\" (UniqueName: \"kubernetes.io/projected/ce4c56eb-3090-42bc-b0ce-a3809301e56b-kube-api-access-495jt\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:57.248505 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:57.248470    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:57.248505 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:57.248507    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5zts\" (UID: \"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:57.248717 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:57.248650    2536 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 02 20:02:57.248753 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:02:57.248720    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert podName:ce4c56eb-3090-42bc-b0ce-a3809301e56b nodeName:}" failed. No retries permitted until 2026-07-02 20:02:58.248701729 +0000 UTC m=+390.396470487 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert") pod "llmisvc-controller-manager-777c6bd7d5-rszpf" (UID: "ce4c56eb-3090-42bc-b0ce-a3809301e56b") : secret "llmisvc-webhook-server-cert" not found
Jul 02 20:02:57.251190 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:57.251169    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d39fc83-8f0b-42e4-b01f-19cb107c4a4b-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5zts\" (UID: \"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:57.465645 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:57.465610    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:02:57.641432 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:57.641411    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"]
Jul 02 20:02:57.643570 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:02:57.643549    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d39fc83_8f0b_42e4_b01f_19cb107c4a4b.slice/crio-792d34509c9b2393fdbdfbde68ac7c5f228e2317d94c11e173d32159903f9b7a WatchSource:0}: Error finding container 792d34509c9b2393fdbdfbde68ac7c5f228e2317d94c11e173d32159903f9b7a: Status 404 returned error can't find the container with id 792d34509c9b2393fdbdfbde68ac7c5f228e2317d94c11e173d32159903f9b7a
Jul 02 20:02:57.825161 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:57.825077    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts" event={"ID":"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b","Type":"ContainerStarted","Data":"792d34509c9b2393fdbdfbde68ac7c5f228e2317d94c11e173d32159903f9b7a"}
Jul 02 20:02:58.258054 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:58.258009    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:58.260860 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:58.260837    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce4c56eb-3090-42bc-b0ce-a3809301e56b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rszpf\" (UID: \"ce4c56eb-3090-42bc-b0ce-a3809301e56b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:58.406090 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:58.406052    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:02:58.579024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:58.578896    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"]
Jul 02 20:02:58.582569 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:02:58.582481    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podce4c56eb_3090_42bc_b0ce_a3809301e56b.slice/crio-16754c5488c1a2522c5659d3e2eb1bfef8e498f7a9d2c47e2b70cbad7e5127c1 WatchSource:0}: Error finding container 16754c5488c1a2522c5659d3e2eb1bfef8e498f7a9d2c47e2b70cbad7e5127c1: Status 404 returned error can't find the container with id 16754c5488c1a2522c5659d3e2eb1bfef8e498f7a9d2c47e2b70cbad7e5127c1
Jul 02 20:02:58.833088 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:02:58.832571    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf" event={"ID":"ce4c56eb-3090-42bc-b0ce-a3809301e56b","Type":"ContainerStarted","Data":"16754c5488c1a2522c5659d3e2eb1bfef8e498f7a9d2c47e2b70cbad7e5127c1"}
Jul 02 20:03:01.085047 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.084997    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:03:01.085590 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.085087    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:03:01.806877 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.806849    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hxpt9"
Jul 02 20:03:01.808955 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.808932    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9jffl"
Jul 02 20:03:01.846649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.846455    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s" event={"ID":"942b340c-b31e-4c63-9acc-466cef257a10","Type":"ContainerStarted","Data":"1147cd13c26e3c6817095d937ce4fdac33b4f07dee7e640de6a4f317234459bd"}
Jul 02 20:03:01.846649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.846589    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:03:01.848546 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.848521    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts" event={"ID":"8d39fc83-8f0b-42e4-b01f-19cb107c4a4b","Type":"ContainerStarted","Data":"c837170387514aec469f1d89a1e1c21bd17bbfdad7f112eeb1f03e85f968c619"}
Jul 02 20:03:01.848694 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.848578    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:03:01.887373 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:01.887323    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s" podStartSLOduration=1.579603191 podStartE2EDuration="14.887304401s" totalImagesPullingTime="13.30770121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:02:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:03:01.886266051 +0000 UTC m=+394.034034854" watchObservedRunningTime="2026-07-02 20:03:01.887304401 +0000 UTC m=+394.035073181"
Jul 02 20:03:02.722939 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.722874    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts" podStartSLOduration=3.282894407 podStartE2EDuration="6.722852475s" totalImagesPullingTime="3.439958068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:03:01.910802931 +0000 UTC m=+394.058571724" watchObservedRunningTime="2026-07-02 20:03:02.722852475 +0000 UTC m=+394.870621255"
Jul 02 20:03:02.724556 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.724531    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"]
Jul 02 20:03:02.729206 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.729186    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.740363 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.740339    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xt9pk\""
Jul 02 20:03:02.770626 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.770596    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"]
Jul 02 20:03:02.798430 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798399    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798430 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798430    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwqq\" (UniqueName: \"kubernetes.io/projected/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-kube-api-access-cpwqq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798465    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798489    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798563    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798595    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798619    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798816 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798635    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.798816 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.798693    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.854382 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.854356    2536 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-55q4s container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 02 20:03:02.854484 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.854412    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s" podUID="942b340c-b31e-4c63-9acc-466cef257a10" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 20:03:02.899894 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.899870    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.899907    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.899928    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.899951    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.899979    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900003    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900031    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900067    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900091    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwqq\" (UniqueName: \"kubernetes.io/projected/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-kube-api-access-cpwqq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900459 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900409    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900650    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900672    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900807 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900744    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.900902 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.900880    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.902327 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.902301    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.902513 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.902494    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.909177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.909154    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:02.910997 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:02.910977    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwqq\" (UniqueName: \"kubernetes.io/projected/b6201a6e-84ee-45a8-9f06-4c04bce06bcb-kube-api-access-cpwqq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zwpwt\" (UID: \"b6201a6e-84ee-45a8-9f06-4c04bce06bcb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:03.041387 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:03.041315    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:03.177657 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:03.177634    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"]
Jul 02 20:03:03.179634 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:03:03.179609    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6201a6e_84ee_45a8_9f06_4c04bce06bcb.slice/crio-36cfce9543c0695c3f079c050a219f9a926dc27cb65e17100cc81e8d366fa59e WatchSource:0}: Error finding container 36cfce9543c0695c3f079c050a219f9a926dc27cb65e17100cc81e8d366fa59e: Status 404 returned error can't find the container with id 36cfce9543c0695c3f079c050a219f9a926dc27cb65e17100cc81e8d366fa59e
Jul 02 20:03:03.858442 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:03.858410    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf" event={"ID":"ce4c56eb-3090-42bc-b0ce-a3809301e56b","Type":"ContainerStarted","Data":"fba90b02cab73210c9cb7d97a56b4c002f46e2e79096c9a6bb6b3a17a87ac8b9"}
Jul 02 20:03:03.858879 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:03.858524    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:03:03.859661 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:03.859634    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt" event={"ID":"b6201a6e-84ee-45a8-9f06-4c04bce06bcb","Type":"ContainerStarted","Data":"36cfce9543c0695c3f079c050a219f9a926dc27cb65e17100cc81e8d366fa59e"}
Jul 02 20:03:03.879945 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:03.879906    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf" podStartSLOduration=3.629095824 podStartE2EDuration="7.879892501s" totalImagesPullingTime="4.250796677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:03:03.879420142 +0000 UTC m=+396.027188940" watchObservedRunningTime="2026-07-02 20:03:03.879892501 +0000 UTC m=+396.027661280"
Jul 02 20:03:04.626103 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:04.626067    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"]
Jul 02 20:03:04.626480 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:04.626376    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s" podUID="942b340c-b31e-4c63-9acc-466cef257a10" containerName="discovery" containerID="cri-o://1147cd13c26e3c6817095d937ce4fdac33b4f07dee7e640de6a4f317234459bd" gracePeriod=30
Jul 02 20:03:04.627126 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:04.627104    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:03:04.866934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:04.866656    2536 generic.go:356] "Generic (PLEG): container finished" podID="942b340c-b31e-4c63-9acc-466cef257a10" containerID="1147cd13c26e3c6817095d937ce4fdac33b4f07dee7e640de6a4f317234459bd" exitCode=0
Jul 02 20:03:04.866934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:04.866722    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s" event={"ID":"942b340c-b31e-4c63-9acc-466cef257a10","Type":"ContainerDied","Data":"1147cd13c26e3c6817095d937ce4fdac33b4f07dee7e640de6a4f317234459bd"}
Jul 02 20:03:04.883908 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:04.883887    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:03:05.017341 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.017313    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-kubeconfig\") pod \"942b340c-b31e-4c63-9acc-466cef257a10\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") "
Jul 02 20:03:05.017533 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.017358    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-dns-cert\") pod \"942b340c-b31e-4c63-9acc-466cef257a10\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") "
Jul 02 20:03:05.017533 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.017412    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/942b340c-b31e-4c63-9acc-466cef257a10-local-certs\") pod \"942b340c-b31e-4c63-9acc-466cef257a10\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") "
Jul 02 20:03:05.017533 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.017451    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-istio-token\") pod \"942b340c-b31e-4c63-9acc-466cef257a10\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") "
Jul 02 20:03:05.017533 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.017478    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5w92\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-kube-api-access-m5w92\") pod \"942b340c-b31e-4c63-9acc-466cef257a10\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") "
Jul 02 20:03:05.017533 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.017510    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-cacerts\") pod \"942b340c-b31e-4c63-9acc-466cef257a10\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") "
Jul 02 20:03:05.017791 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.017579    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-ca-configmap\") pod \"942b340c-b31e-4c63-9acc-466cef257a10\" (UID: \"942b340c-b31e-4c63-9acc-466cef257a10\") "
Jul 02 20:03:05.018237 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.018091    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "942b340c-b31e-4c63-9acc-466cef257a10" (UID: "942b340c-b31e-4c63-9acc-466cef257a10"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:03:05.020395 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.020350    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-cacerts" (OuterVolumeSpecName: "cacerts") pod "942b340c-b31e-4c63-9acc-466cef257a10" (UID: "942b340c-b31e-4c63-9acc-466cef257a10"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:03:05.020535 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.020500    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "942b340c-b31e-4c63-9acc-466cef257a10" (UID: "942b340c-b31e-4c63-9acc-466cef257a10"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:03:05.020657 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.020628    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/942b340c-b31e-4c63-9acc-466cef257a10-local-certs" (OuterVolumeSpecName: "local-certs") pod "942b340c-b31e-4c63-9acc-466cef257a10" (UID: "942b340c-b31e-4c63-9acc-466cef257a10"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:03:05.020798 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.020772    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-istio-token" (OuterVolumeSpecName: "istio-token") pod "942b340c-b31e-4c63-9acc-466cef257a10" (UID: "942b340c-b31e-4c63-9acc-466cef257a10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:03:05.021170 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.021125    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "942b340c-b31e-4c63-9acc-466cef257a10" (UID: "942b340c-b31e-4c63-9acc-466cef257a10"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:03:05.021242 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.021220    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-kube-api-access-m5w92" (OuterVolumeSpecName: "kube-api-access-m5w92") pod "942b340c-b31e-4c63-9acc-466cef257a10" (UID: "942b340c-b31e-4c63-9acc-466cef257a10"). InnerVolumeSpecName "kube-api-access-m5w92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:03:05.119007 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.118974    2536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-istio-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:05.119007 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.119009    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5w92\" (UniqueName: \"kubernetes.io/projected/942b340c-b31e-4c63-9acc-466cef257a10-kube-api-access-m5w92\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:05.119205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.119018    2536 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-cacerts\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:05.119205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.119027    2536 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-ca-configmap\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:05.119205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.119041    2536 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-kubeconfig\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:05.119205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.119050    2536 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/942b340c-b31e-4c63-9acc-466cef257a10-istio-csr-dns-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:05.119205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.119058    2536 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/942b340c-b31e-4c63-9acc-466cef257a10-local-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:05.872622 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.872593    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"
Jul 02 20:03:05.873035 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.872596    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-55q4s" event={"ID":"942b340c-b31e-4c63-9acc-466cef257a10","Type":"ContainerDied","Data":"4b2c2870f6362d231619bae33e2425964fc7d3720091a0ea9f371e24fee9a08f"}
Jul 02 20:03:05.873035 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.872737    2536 scope.go:117] "RemoveContainer" containerID="1147cd13c26e3c6817095d937ce4fdac33b4f07dee7e640de6a4f317234459bd"
Jul 02 20:03:05.903201 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.903170    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"]
Jul 02 20:03:05.906612 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:05.906581    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-55q4s"]
Jul 02 20:03:06.369397 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:06.369364    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="942b340c-b31e-4c63-9acc-466cef257a10" path="/var/lib/kubelet/pods/942b340c-b31e-4c63-9acc-466cef257a10/volumes"
Jul 02 20:03:06.570153 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:06.570121    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:03:06.570241 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:06.570188    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:03:06.570241 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:06.570236    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:03:06.879213 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:06.879130    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt" event={"ID":"b6201a6e-84ee-45a8-9f06-4c04bce06bcb","Type":"ContainerStarted","Data":"5c988160638ac59601a4e752960c71f1092566c06226460a6f3fd777f3538133"}
Jul 02 20:03:06.929162 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:06.929100    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt" podStartSLOduration=1.5410087689999998 podStartE2EDuration="4.929077764s" totalImagesPullingTime="3.388068995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:03:06.928937979 +0000 UTC m=+399.076706776" watchObservedRunningTime="2026-07-02 20:03:06.929077764 +0000 UTC m=+399.076846546"
Jul 02 20:03:07.041600 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:07.041568    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:07.042938 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:07.042915    2536 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body=
Jul 02 20:03:07.043043 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:07.042969    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt" podUID="b6201a6e-84ee-45a8-9f06-4c04bce06bcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused"
Jul 02 20:03:08.045619 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:08.045593    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:08.888171 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:08.888141    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:08.889326 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:08.889310    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zwpwt"
Jul 02 20:03:32.857449 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:32.857420    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5zts"
Jul 02 20:03:34.869386 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:34.869356    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rszpf"
Jul 02 20:03:46.282202 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.282172    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gxvvk"]
Jul 02 20:03:46.282584 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.282453    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="942b340c-b31e-4c63-9acc-466cef257a10" containerName="discovery"
Jul 02 20:03:46.282584 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.282465    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="942b340c-b31e-4c63-9acc-466cef257a10" containerName="discovery"
Jul 02 20:03:46.282584 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.282523    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="942b340c-b31e-4c63-9acc-466cef257a10" containerName="discovery"
Jul 02 20:03:46.297392 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.297366    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk"
Jul 02 20:03:46.300922 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.300898    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 02 20:03:46.301393 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.301375    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-wngts\""
Jul 02 20:03:46.301967 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.301946    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 02 20:03:46.321206 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.321181    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gxvvk"]
Jul 02 20:03:46.329643 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.329624    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cssqt\" (UniqueName: \"kubernetes.io/projected/c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8-kube-api-access-cssqt\") pod \"kuadrant-operator-catalog-gxvvk\" (UID: \"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8\") " pod="kuadrant-system/kuadrant-operator-catalog-gxvvk"
Jul 02 20:03:46.430605 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.430536    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cssqt\" (UniqueName: \"kubernetes.io/projected/c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8-kube-api-access-cssqt\") pod \"kuadrant-operator-catalog-gxvvk\" (UID: \"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8\") " pod="kuadrant-system/kuadrant-operator-catalog-gxvvk"
Jul 02 20:03:46.441002 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.440973    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cssqt\" (UniqueName: \"kubernetes.io/projected/c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8-kube-api-access-cssqt\") pod \"kuadrant-operator-catalog-gxvvk\" (UID: \"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8\") " pod="kuadrant-system/kuadrant-operator-catalog-gxvvk"
Jul 02 20:03:46.564990 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.564921    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gxvvk"]
Jul 02 20:03:46.565184 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.565172    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk"
Jul 02 20:03:46.696326 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.696257    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gxvvk"]
Jul 02 20:03:46.698446 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:03:46.698422    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31c7c2f_492a_4f8d_a6b2_a1c205f5ecb8.slice/crio-d700762fb3559ab961f745001c49e1234a69b6ba20c0ee45cb9b2fdd00019109 WatchSource:0}: Error finding container d700762fb3559ab961f745001c49e1234a69b6ba20c0ee45cb9b2fdd00019109: Status 404 returned error can't find the container with id d700762fb3559ab961f745001c49e1234a69b6ba20c0ee45cb9b2fdd00019109
Jul 02 20:03:46.777716 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.777686    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r7w9z"]
Jul 02 20:03:46.782081 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.782062    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:46.796253 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.796233    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r7w9z"]
Jul 02 20:03:46.834421 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.834361    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wddp\" (UniqueName: \"kubernetes.io/projected/bcc963fb-935d-4f49-9019-f87ded78cbdf-kube-api-access-4wddp\") pod \"kuadrant-operator-catalog-r7w9z\" (UID: \"bcc963fb-935d-4f49-9019-f87ded78cbdf\") " pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:46.935385 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.935354    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wddp\" (UniqueName: \"kubernetes.io/projected/bcc963fb-935d-4f49-9019-f87ded78cbdf-kube-api-access-4wddp\") pod \"kuadrant-operator-catalog-r7w9z\" (UID: \"bcc963fb-935d-4f49-9019-f87ded78cbdf\") " pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:46.945372 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:46.945334    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wddp\" (UniqueName: \"kubernetes.io/projected/bcc963fb-935d-4f49-9019-f87ded78cbdf-kube-api-access-4wddp\") pod \"kuadrant-operator-catalog-r7w9z\" (UID: \"bcc963fb-935d-4f49-9019-f87ded78cbdf\") " pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:47.038805 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:47.038769    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk" event={"ID":"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8","Type":"ContainerStarted","Data":"d700762fb3559ab961f745001c49e1234a69b6ba20c0ee45cb9b2fdd00019109"}
Jul 02 20:03:47.090177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:47.090089    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:47.217687 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:47.217664    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r7w9z"]
Jul 02 20:03:47.219729 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:03:47.219705    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc963fb_935d_4f49_9019_f87ded78cbdf.slice/crio-45eb6216fb069c67f10a159f7d71439e36085c301231c7f3efd11a0fd9a5f064 WatchSource:0}: Error finding container 45eb6216fb069c67f10a159f7d71439e36085c301231c7f3efd11a0fd9a5f064: Status 404 returned error can't find the container with id 45eb6216fb069c67f10a159f7d71439e36085c301231c7f3efd11a0fd9a5f064
Jul 02 20:03:48.044976 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:48.044936    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z" event={"ID":"bcc963fb-935d-4f49-9019-f87ded78cbdf","Type":"ContainerStarted","Data":"45eb6216fb069c67f10a159f7d71439e36085c301231c7f3efd11a0fd9a5f064"}
Jul 02 20:03:49.051213 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.051114    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk" event={"ID":"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8","Type":"ContainerStarted","Data":"37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d"}
Jul 02 20:03:49.051213 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.051168    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk" podUID="c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8" containerName="registry-server" containerID="cri-o://37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d" gracePeriod=2
Jul 02 20:03:49.052734 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.052703    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z" event={"ID":"bcc963fb-935d-4f49-9019-f87ded78cbdf","Type":"ContainerStarted","Data":"9ee95f601212568711ab3d48f675e050a5ec8bceafda3826e7fc2c4c7517fa65"}
Jul 02 20:03:49.068751 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.068712    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk" podStartSLOduration=1.106801372 podStartE2EDuration="3.068697426s" totalImagesPullingTime="1.961896054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:03:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:03:49.068102942 +0000 UTC m=+441.215871731" watchObservedRunningTime="2026-07-02 20:03:49.068697426 +0000 UTC m=+441.216466205"
Jul 02 20:03:49.085260 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.085203    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z" podStartSLOduration=1.6280640499999999 podStartE2EDuration="3.08518882s" totalImagesPullingTime="1.45712477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:03:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:03:49.08376789 +0000 UTC m=+441.231536669" watchObservedRunningTime="2026-07-02 20:03:49.08518882 +0000 UTC m=+441.232957617"
Jul 02 20:03:49.293696 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.293669    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk"
Jul 02 20:03:49.353357 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.353277    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cssqt\" (UniqueName: \"kubernetes.io/projected/c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8-kube-api-access-cssqt\") pod \"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8\" (UID: \"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8\") "
Jul 02 20:03:49.355422 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.355393    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8-kube-api-access-cssqt" (OuterVolumeSpecName: "kube-api-access-cssqt") pod "c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8" (UID: "c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8"). InnerVolumeSpecName "kube-api-access-cssqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:03:49.454062 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:49.454032    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cssqt\" (UniqueName: \"kubernetes.io/projected/c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8-kube-api-access-cssqt\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:03:50.057878 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.057797    2536 generic.go:356] "Generic (PLEG): container finished" podID="c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8" containerID="37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d" exitCode=0
Jul 02 20:03:50.057878 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.057856    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk"
Jul 02 20:03:50.058322 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.057880    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk" event={"ID":"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8","Type":"ContainerDied","Data":"37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d"}
Jul 02 20:03:50.058322 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.057927    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gxvvk" event={"ID":"c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8","Type":"ContainerDied","Data":"d700762fb3559ab961f745001c49e1234a69b6ba20c0ee45cb9b2fdd00019109"}
Jul 02 20:03:50.058322 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.057946    2536 scope.go:117] "RemoveContainer" containerID="37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d"
Jul 02 20:03:50.066712 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.066695    2536 scope.go:117] "RemoveContainer" containerID="37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d"
Jul 02 20:03:50.066964 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:03:50.066946    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d\": container with ID starting with 37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d not found: ID does not exist" containerID="37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d"
Jul 02 20:03:50.067049 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.066976    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d"} err="failed to get container status \"37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d\": rpc error: code = NotFound desc = could not find container \"37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d\": container with ID starting with 37f8a371ccc2b0ba5369d98ce22f622c445086dfecb4d595c01e6da09895a18d not found: ID does not exist"
Jul 02 20:03:50.077739 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.077716    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gxvvk"]
Jul 02 20:03:50.083902 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.083884    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gxvvk"]
Jul 02 20:03:50.368280 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:50.368210    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8" path="/var/lib/kubelet/pods/c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8/volumes"
Jul 02 20:03:57.090839 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:57.090804    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:57.091348 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:57.090881    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:57.113038 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:57.113017    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:03:58.112476 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:03:58.112450    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r7w9z"
Jul 02 20:04:19.320762 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.320705    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"]
Jul 02 20:04:19.321197 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.321029    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8" containerName="registry-server"
Jul 02 20:04:19.321197 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.321041    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8" containerName="registry-server"
Jul 02 20:04:19.321197 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.321111    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="c31c7c2f-492a-4f8d-a6b2-a1c205f5ecb8" containerName="registry-server"
Jul 02 20:04:19.329201 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.329184    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"
Jul 02 20:04:19.332870 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.332850    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-j5jfd\""
Jul 02 20:04:19.340403 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.340381    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"]
Jul 02 20:04:19.494372 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.494335    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tcjk\" (UniqueName: \"kubernetes.io/projected/ae94fc96-285f-4ec4-9380-76e86b131035-kube-api-access-9tcjk\") pod \"authorino-operator-75cc8d5cb7-dsxjg\" (UID: \"ae94fc96-285f-4ec4-9380-76e86b131035\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"
Jul 02 20:04:19.595477 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.595379    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tcjk\" (UniqueName: \"kubernetes.io/projected/ae94fc96-285f-4ec4-9380-76e86b131035-kube-api-access-9tcjk\") pod \"authorino-operator-75cc8d5cb7-dsxjg\" (UID: \"ae94fc96-285f-4ec4-9380-76e86b131035\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"
Jul 02 20:04:19.608627 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.608600    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tcjk\" (UniqueName: \"kubernetes.io/projected/ae94fc96-285f-4ec4-9380-76e86b131035-kube-api-access-9tcjk\") pod \"authorino-operator-75cc8d5cb7-dsxjg\" (UID: \"ae94fc96-285f-4ec4-9380-76e86b131035\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"
Jul 02 20:04:19.638317 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.638271    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"
Jul 02 20:04:19.771578 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:19.771547    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"]
Jul 02 20:04:19.774481 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:04:19.774451    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae94fc96_285f_4ec4_9380_76e86b131035.slice/crio-c8d9c29d91ac6b7a01d3d4b2943e82096be5698064c85eb836830250889ebcd4 WatchSource:0}: Error finding container c8d9c29d91ac6b7a01d3d4b2943e82096be5698064c85eb836830250889ebcd4: Status 404 returned error can't find the container with id c8d9c29d91ac6b7a01d3d4b2943e82096be5698064c85eb836830250889ebcd4
Jul 02 20:04:20.175337 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:20.175273    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg" event={"ID":"ae94fc96-285f-4ec4-9380-76e86b131035","Type":"ContainerStarted","Data":"c8d9c29d91ac6b7a01d3d4b2943e82096be5698064c85eb836830250889ebcd4"}
Jul 02 20:04:23.189694 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.189663    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg" event={"ID":"ae94fc96-285f-4ec4-9380-76e86b131035","Type":"ContainerStarted","Data":"8f49b13a2b30bfc623bd9e49d463ab2b8a429a3d45b397cac7348fbcd8fde1f2"}
Jul 02 20:04:23.190090 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.189790    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"
Jul 02 20:04:23.236223 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.236176    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg" podStartSLOduration=1.862301421 podStartE2EDuration="4.236155291s" totalImagesPullingTime="2.37385387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:04:23.227572134 +0000 UTC m=+475.375340903" watchObservedRunningTime="2026-07-02 20:04:23.236155291 +0000 UTC m=+475.383924071"
Jul 02 20:04:23.254548 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.254510    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"]
Jul 02 20:04:23.257836 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.257819    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.262465 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.262449    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-26xj4\""
Jul 02 20:04:23.276125 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.276104    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"]
Jul 02 20:04:23.431338 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.431272    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.431508 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.431400    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2grs5\" (UniqueName: \"kubernetes.io/projected/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-kube-api-access-2grs5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.532335 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.532221    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2grs5\" (UniqueName: \"kubernetes.io/projected/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-kube-api-access-2grs5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.532335 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.532304    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.532689 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.532670    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.546339 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.546309    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2grs5\" (UniqueName: \"kubernetes.io/projected/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-kube-api-access-2grs5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.572397 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.572367    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:23.730135 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:23.730103    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"]
Jul 02 20:04:23.733845 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:04:23.733818    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21cc5188_c801_4ab0_b7a6_3695a0ca2e4f.slice/crio-5bcace4590d0092d8caf5427162c8c75d187a7373066897676b4a9d164e5fc76 WatchSource:0}: Error finding container 5bcace4590d0092d8caf5427162c8c75d187a7373066897676b4a9d164e5fc76: Status 404 returned error can't find the container with id 5bcace4590d0092d8caf5427162c8c75d187a7373066897676b4a9d164e5fc76
Jul 02 20:04:24.194610 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:24.194570    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc" event={"ID":"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f","Type":"ContainerStarted","Data":"5bcace4590d0092d8caf5427162c8c75d187a7373066897676b4a9d164e5fc76"}
Jul 02 20:04:25.391757 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.391722    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"]
Jul 02 20:04:25.395186 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.395164    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"
Jul 02 20:04:25.398056 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.398035    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 02 20:04:25.398056 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.398045    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-s78pb\""
Jul 02 20:04:25.409067 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.409046    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"]
Jul 02 20:04:25.551962 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.551926    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8rhc\" (UniqueName: \"kubernetes.io/projected/17026fea-7580-4627-8aa5-c9e92d1d3567-kube-api-access-g8rhc\") pod \"dns-operator-controller-manager-864dd78c4c-z59sj\" (UID: \"17026fea-7580-4627-8aa5-c9e92d1d3567\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"
Jul 02 20:04:25.653447 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.653364    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8rhc\" (UniqueName: \"kubernetes.io/projected/17026fea-7580-4627-8aa5-c9e92d1d3567-kube-api-access-g8rhc\") pod \"dns-operator-controller-manager-864dd78c4c-z59sj\" (UID: \"17026fea-7580-4627-8aa5-c9e92d1d3567\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"
Jul 02 20:04:25.662581 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.662547    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8rhc\" (UniqueName: \"kubernetes.io/projected/17026fea-7580-4627-8aa5-c9e92d1d3567-kube-api-access-g8rhc\") pod \"dns-operator-controller-manager-864dd78c4c-z59sj\" (UID: \"17026fea-7580-4627-8aa5-c9e92d1d3567\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"
Jul 02 20:04:25.708060 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.708027    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"
Jul 02 20:04:25.838033 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:25.838000    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"]
Jul 02 20:04:25.840871 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:04:25.840845    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17026fea_7580_4627_8aa5_c9e92d1d3567.slice/crio-31c1ed1375355be285f2739242bcd0db4611e7ec7e005c5b3e3de289e2d034af WatchSource:0}: Error finding container 31c1ed1375355be285f2739242bcd0db4611e7ec7e005c5b3e3de289e2d034af: Status 404 returned error can't find the container with id 31c1ed1375355be285f2739242bcd0db4611e7ec7e005c5b3e3de289e2d034af
Jul 02 20:04:26.205682 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:26.205647    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj" event={"ID":"17026fea-7580-4627-8aa5-c9e92d1d3567","Type":"ContainerStarted","Data":"31c1ed1375355be285f2739242bcd0db4611e7ec7e005c5b3e3de289e2d034af"}
Jul 02 20:04:28.024115 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.024086    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"]
Jul 02 20:04:28.027755 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.027702    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:28.031075 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.031048    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-79tkc\""
Jul 02 20:04:28.038967 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.038940    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"]
Jul 02 20:04:28.176421 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.176383    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbzf5\" (UniqueName: \"kubernetes.io/projected/2075526f-6d6b-4171-8a75-814b18892028-kube-api-access-fbzf5\") pod \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" (UID: \"2075526f-6d6b-4171-8a75-814b18892028\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:28.277853 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.277764    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbzf5\" (UniqueName: \"kubernetes.io/projected/2075526f-6d6b-4171-8a75-814b18892028-kube-api-access-fbzf5\") pod \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" (UID: \"2075526f-6d6b-4171-8a75-814b18892028\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:28.287476 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.287445    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbzf5\" (UniqueName: \"kubernetes.io/projected/2075526f-6d6b-4171-8a75-814b18892028-kube-api-access-fbzf5\") pod \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" (UID: \"2075526f-6d6b-4171-8a75-814b18892028\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:28.344717 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.344673    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-79tkc\""
Jul 02 20:04:28.352115 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:28.352084    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:29.318272 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:29.318250    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 02 20:04:29.458514 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:29.458479    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"]
Jul 02 20:04:29.461909 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:04:29.461880    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2075526f_6d6b_4171_8a75_814b18892028.slice/crio-2f6f85ef3a1e8d19a8bb0f3a327bdb5ca233248bb78406f4955a78e49fded804 WatchSource:0}: Error finding container 2f6f85ef3a1e8d19a8bb0f3a327bdb5ca233248bb78406f4955a78e49fded804: Status 404 returned error can't find the container with id 2f6f85ef3a1e8d19a8bb0f3a327bdb5ca233248bb78406f4955a78e49fded804
Jul 02 20:04:30.227685 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:30.227649    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9" event={"ID":"2075526f-6d6b-4171-8a75-814b18892028","Type":"ContainerStarted","Data":"2f6f85ef3a1e8d19a8bb0f3a327bdb5ca233248bb78406f4955a78e49fded804"}
Jul 02 20:04:30.229679 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:30.229650    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj" event={"ID":"17026fea-7580-4627-8aa5-c9e92d1d3567","Type":"ContainerStarted","Data":"e60d917914fb807c749bb187454d79ec06713ba93592406211f8f8d6ae5cda25"}
Jul 02 20:04:30.229838 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:30.229790    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"
Jul 02 20:04:30.231855 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:30.231826    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc" event={"ID":"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f","Type":"ContainerStarted","Data":"73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425"}
Jul 02 20:04:30.232085 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:30.232068    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:30.250631 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:30.250565    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj" podStartSLOduration=1.77850966 podStartE2EDuration="5.250545856s" totalImagesPullingTime="3.472036196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:04:30.24927565 +0000 UTC m=+482.397044426" watchObservedRunningTime="2026-07-02 20:04:30.250545856 +0000 UTC m=+482.398314638"
Jul 02 20:04:30.271094 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:30.271028    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc" podStartSLOduration=1.681713451 podStartE2EDuration="7.27100868s" totalImagesPullingTime="5.589295229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:04:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:04:30.269366222 +0000 UTC m=+482.417135002" watchObservedRunningTime="2026-07-02 20:04:30.27100868 +0000 UTC m=+482.418777460"
Jul 02 20:04:32.243578 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:32.243548    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9" event={"ID":"2075526f-6d6b-4171-8a75-814b18892028","Type":"ContainerStarted","Data":"27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e"}
Jul 02 20:04:32.243926 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:32.243677    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:32.268097 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:32.268046    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9" podStartSLOduration=2.487516476 podStartE2EDuration="4.268029547s" totalImagesPullingTime="1.780513071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:04:32.265502889 +0000 UTC m=+484.413271685" watchObservedRunningTime="2026-07-02 20:04:32.268029547 +0000 UTC m=+484.415798327"
Jul 02 20:04:34.197124 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:34.197095    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dsxjg"
Jul 02 20:04:41.237771 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:41.237740    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:41.238279 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:41.237810    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z59sj"
Jul 02 20:04:43.229463 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.229425    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"]
Jul 02 20:04:43.229870 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.229607    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" containerName="manager" containerID="cri-o://73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425" gracePeriod=2
Jul 02 20:04:43.250636 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.250608    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.251614 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.251594    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"]
Jul 02 20:04:43.257109 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.257081    2536 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\": pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.260070 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.260047    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.283658 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.283637    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"]
Jul 02 20:04:43.289843 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.289820    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9" podUID="2075526f-6d6b-4171-8a75-814b18892028" containerName="manager" containerID="cri-o://27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e" gracePeriod=2
Jul 02 20:04:43.308676 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.308647    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"]
Jul 02 20:04:43.319696 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.319625    2536 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2075526f-6d6b-4171-8a75-814b18892028\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-02T20:04:43Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-02T20:04:43Z\\\",\\\"message\\\":null,\\\"reason\\\":null,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"},\\\"containerID\\\":\\\"cri-o://27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e\\\",\\\"image\\\":\\\"quay.io/kuadrant/limitador-operator:v0.17.1\\\",\\\"imageID\\\":\\\"quay.io/kuadrant/limitador-operator@sha256:3707fe24c6052d5adf240e12d34110706f0ac0877373830b67daab7c9a5c380d\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"manager\\\",\\\"ready\\\":true,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"300Mi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"200m\\\",\\\"memory\\\":\\\"200Mi\\\"}},\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-02T20:04:31Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000680000],\\\"uid\\\":1000680000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-fbzf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-bn9p9\": pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.322144 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.322108    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="2075526f-6d6b-4171-8a75-814b18892028" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.322246 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.322186    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"]
Jul 02 20:04:43.322539 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.322523    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" containerName="manager"
Jul 02 20:04:43.322607 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.322542    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" containerName="manager"
Jul 02 20:04:43.322607 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.322563    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2075526f-6d6b-4171-8a75-814b18892028" containerName="manager"
Jul 02 20:04:43.322607 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.322571    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="2075526f-6d6b-4171-8a75-814b18892028" containerName="manager"
Jul 02 20:04:43.322747 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.322658    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" containerName="manager"
Jul 02 20:04:43.322747 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.322672    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="2075526f-6d6b-4171-8a75-814b18892028" containerName="manager"
Jul 02 20:04:43.323998 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.323974    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.325555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.325539    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.327953 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.327932    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.335634 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.335612    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"]
Jul 02 20:04:43.338913 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.338895    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"
Jul 02 20:04:43.358017 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.357978    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"]
Jul 02 20:04:43.360042 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.360021    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"]
Jul 02 20:04:43.384167 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.384135    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="2075526f-6d6b-4171-8a75-814b18892028" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.386940 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.386911    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="2075526f-6d6b-4171-8a75-814b18892028" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.389094 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.389072    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.487854 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.487778    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm7rx\" (UniqueName: \"kubernetes.io/projected/6135f634-50ec-4fc2-a539-354ea4d4fefe-kube-api-access-qm7rx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-llnkt\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.487999 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.487928    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g66df\" (UniqueName: \"kubernetes.io/projected/03925802-a366-43e1-9bd7-09031289299a-kube-api-access-g66df\") pod \"limitador-operator-controller-manager-859d7b78bd-hj8dz\" (UID: \"03925802-a366-43e1-9bd7-09031289299a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"
Jul 02 20:04:43.487999 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.487954    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6135f634-50ec-4fc2-a539-354ea4d4fefe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-llnkt\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.526271 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.526251    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.528619 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.528597    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="2075526f-6d6b-4171-8a75-814b18892028" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.531362 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.531341    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.531450 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.531431    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.534195 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.534178    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="2075526f-6d6b-4171-8a75-814b18892028" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:43.536400 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:43.536377    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:43.588692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.588664    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbzf5\" (UniqueName: \"kubernetes.io/projected/2075526f-6d6b-4171-8a75-814b18892028-kube-api-access-fbzf5\") pod \"2075526f-6d6b-4171-8a75-814b18892028\" (UID: \"2075526f-6d6b-4171-8a75-814b18892028\") "
Jul 02 20:04:43.588817 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.588758    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2grs5\" (UniqueName: \"kubernetes.io/projected/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-kube-api-access-2grs5\") pod \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") "
Jul 02 20:04:43.588817 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.588809    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-extensions-socket-volume\") pod \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\" (UID: \"21cc5188-c801-4ab0-b7a6-3695a0ca2e4f\") "
Jul 02 20:04:43.588929 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.588914    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm7rx\" (UniqueName: \"kubernetes.io/projected/6135f634-50ec-4fc2-a539-354ea4d4fefe-kube-api-access-qm7rx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-llnkt\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.589034 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.589013    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g66df\" (UniqueName: \"kubernetes.io/projected/03925802-a366-43e1-9bd7-09031289299a-kube-api-access-g66df\") pod \"limitador-operator-controller-manager-859d7b78bd-hj8dz\" (UID: \"03925802-a366-43e1-9bd7-09031289299a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"
Jul 02 20:04:43.589159 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.589040    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6135f634-50ec-4fc2-a539-354ea4d4fefe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-llnkt\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.589941 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.589913    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6135f634-50ec-4fc2-a539-354ea4d4fefe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-llnkt\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.590037 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.589916    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" (UID: "21cc5188-c801-4ab0-b7a6-3695a0ca2e4f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:04:43.595161 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.591624    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-kube-api-access-2grs5" (OuterVolumeSpecName: "kube-api-access-2grs5") pod "21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" (UID: "21cc5188-c801-4ab0-b7a6-3695a0ca2e4f"). InnerVolumeSpecName "kube-api-access-2grs5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:04:43.595161 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.591673    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2075526f-6d6b-4171-8a75-814b18892028-kube-api-access-fbzf5" (OuterVolumeSpecName: "kube-api-access-fbzf5") pod "2075526f-6d6b-4171-8a75-814b18892028" (UID: "2075526f-6d6b-4171-8a75-814b18892028"). InnerVolumeSpecName "kube-api-access-fbzf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:04:43.598910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.598887    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g66df\" (UniqueName: \"kubernetes.io/projected/03925802-a366-43e1-9bd7-09031289299a-kube-api-access-g66df\") pod \"limitador-operator-controller-manager-859d7b78bd-hj8dz\" (UID: \"03925802-a366-43e1-9bd7-09031289299a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"
Jul 02 20:04:43.598993 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.598887    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm7rx\" (UniqueName: \"kubernetes.io/projected/6135f634-50ec-4fc2-a539-354ea4d4fefe-kube-api-access-qm7rx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-llnkt\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.667426 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.667396    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:43.675455 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.675439    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"
Jul 02 20:04:43.690529 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.690504    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2grs5\" (UniqueName: \"kubernetes.io/projected/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-kube-api-access-2grs5\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:04:43.690529 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.690526    2536 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f-extensions-socket-volume\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:04:43.690693 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.690536    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbzf5\" (UniqueName: \"kubernetes.io/projected/2075526f-6d6b-4171-8a75-814b18892028-kube-api-access-fbzf5\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:04:43.824210 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.824174    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"]
Jul 02 20:04:43.827476 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:04:43.827447    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6135f634_50ec_4fc2_a539_354ea4d4fefe.slice/crio-2cd5b45261e664708b414b067d5f37be566ee3844d05df23a2c6b2bd2eb8494f WatchSource:0}: Error finding container 2cd5b45261e664708b414b067d5f37be566ee3844d05df23a2c6b2bd2eb8494f: Status 404 returned error can't find the container with id 2cd5b45261e664708b414b067d5f37be566ee3844d05df23a2c6b2bd2eb8494f
Jul 02 20:04:43.847085 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:43.847059    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"]
Jul 02 20:04:43.848719 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:04:43.848698    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03925802_a366_43e1_9bd7_09031289299a.slice/crio-72770fdf654fd62d9c613e4ed127ec6229f5a13ffc3520a7ddd636091fe8a7ff WatchSource:0}: Error finding container 72770fdf654fd62d9c613e4ed127ec6229f5a13ffc3520a7ddd636091fe8a7ff: Status 404 returned error can't find the container with id 72770fdf654fd62d9c613e4ed127ec6229f5a13ffc3520a7ddd636091fe8a7ff
Jul 02 20:04:44.295835 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.295795    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz" event={"ID":"03925802-a366-43e1-9bd7-09031289299a","Type":"ContainerStarted","Data":"0ef03d02b03fd8fa0a650262dc0f3c68c59928fe994759e45f506f1fe8d14bfb"}
Jul 02 20:04:44.296279 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.295855    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz" event={"ID":"03925802-a366-43e1-9bd7-09031289299a","Type":"ContainerStarted","Data":"72770fdf654fd62d9c613e4ed127ec6229f5a13ffc3520a7ddd636091fe8a7ff"}
Jul 02 20:04:44.296279 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.295923    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"
Jul 02 20:04:44.297425 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.297402    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt" event={"ID":"6135f634-50ec-4fc2-a539-354ea4d4fefe","Type":"ContainerStarted","Data":"3b9ce02b228e07a9849622df4c12f40760fb97fb6aa38e1ec7d0a33201158af1"}
Jul 02 20:04:44.297530 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.297447    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt" event={"ID":"6135f634-50ec-4fc2-a539-354ea4d4fefe","Type":"ContainerStarted","Data":"2cd5b45261e664708b414b067d5f37be566ee3844d05df23a2c6b2bd2eb8494f"}
Jul 02 20:04:44.297530 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.297469    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:44.298624 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.298606    2536 generic.go:356] "Generic (PLEG): container finished" podID="2075526f-6d6b-4171-8a75-814b18892028" containerID="27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e" exitCode=0
Jul 02 20:04:44.298706 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.298653    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:44.298765 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.298704    2536 scope.go:117] "RemoveContainer" containerID="27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e"
Jul 02 20:04:44.299981 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.299956    2536 generic.go:356] "Generic (PLEG): container finished" podID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" containerID="73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425" exitCode=0
Jul 02 20:04:44.300040 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.300023    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:44.310953 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.310929    2536 scope.go:117] "RemoveContainer" containerID="27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e"
Jul 02 20:04:44.311218 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:44.311196    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e\": container with ID starting with 27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e not found: ID does not exist" containerID="27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e"
Jul 02 20:04:44.311347 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.311223    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e"} err="failed to get container status \"27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e\": rpc error: code = NotFound desc = could not find container \"27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e\": container with ID starting with 27b14ac232c14ccecc8cc2e00f9ff74d0e8afc6a8c890b8c8636ea9ede50e85e not found: ID does not exist"
Jul 02 20:04:44.311347 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.311241    2536 scope.go:117] "RemoveContainer" containerID="73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425"
Jul 02 20:04:44.319177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.319154    2536 scope.go:117] "RemoveContainer" containerID="73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425"
Jul 02 20:04:44.319458 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:44.319440    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425\": container with ID starting with 73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425 not found: ID does not exist" containerID="73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425"
Jul 02 20:04:44.319519 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.319466    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425"} err="failed to get container status \"73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425\": rpc error: code = NotFound desc = could not find container \"73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425\": container with ID starting with 73f5490b9156bbe1d776fd403ebd24f578a4bf6c74381beae1556849fc1cf425 not found: ID does not exist"
Jul 02 20:04:44.320464 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.320430    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz" podStartSLOduration=1.320416672 podStartE2EDuration="1.320416672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:04:44.319144899 +0000 UTC m=+496.466913679" watchObservedRunningTime="2026-07-02 20:04:44.320416672 +0000 UTC m=+496.468185430"
Jul 02 20:04:44.324648 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:44.324626    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="2075526f-6d6b-4171-8a75-814b18892028" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:44.326509 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:44.326493    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:44.358444 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.358400    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt" podStartSLOduration=1.358383339 podStartE2EDuration="1.358383339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:04:44.356038922 +0000 UTC m=+496.503807701" watchObservedRunningTime="2026-07-02 20:04:44.358383339 +0000 UTC m=+496.506152118"
Jul 02 20:04:44.358989 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:44.358953    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bn9p9\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="2075526f-6d6b-4171-8a75-814b18892028" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bn9p9"
Jul 02 20:04:44.361209 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:04:44.361187    2536 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rs2bc\" is forbidden: User \"system:node:ip-10-0-142-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-101.ec2.internal' and this object" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rs2bc"
Jul 02 20:04:44.368791 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.368770    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2075526f-6d6b-4171-8a75-814b18892028" path="/var/lib/kubelet/pods/2075526f-6d6b-4171-8a75-814b18892028/volumes"
Jul 02 20:04:44.369102 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:44.369089    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21cc5188-c801-4ab0-b7a6-3695a0ca2e4f" path="/var/lib/kubelet/pods/21cc5188-c801-4ab0-b7a6-3695a0ca2e4f/volumes"
Jul 02 20:04:55.308302 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:55.308266    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:04:55.308681 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:04:55.308375    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hj8dz"
Jul 02 20:05:15.282932 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.282893    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"]
Jul 02 20:05:15.283449 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.283175    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt" podUID="6135f634-50ec-4fc2-a539-354ea4d4fefe" containerName="manager" containerID="cri-o://3b9ce02b228e07a9849622df4c12f40760fb97fb6aa38e1ec7d0a33201158af1" gracePeriod=10
Jul 02 20:05:15.307025 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.306988    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt" podUID="6135f634-50ec-4fc2-a539-354ea4d4fefe" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.29:8081/readyz\": dial tcp 10.133.0.29:8081: connect: connection refused"
Jul 02 20:05:15.429696 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.429655    2536 generic.go:356] "Generic (PLEG): container finished" podID="6135f634-50ec-4fc2-a539-354ea4d4fefe" containerID="3b9ce02b228e07a9849622df4c12f40760fb97fb6aa38e1ec7d0a33201158af1" exitCode=0
Jul 02 20:05:15.429885 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.429755    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt" event={"ID":"6135f634-50ec-4fc2-a539-354ea4d4fefe","Type":"ContainerDied","Data":"3b9ce02b228e07a9849622df4c12f40760fb97fb6aa38e1ec7d0a33201158af1"}
Jul 02 20:05:15.532244 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.532222    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:05:15.641421 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.641396    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm7rx\" (UniqueName: \"kubernetes.io/projected/6135f634-50ec-4fc2-a539-354ea4d4fefe-kube-api-access-qm7rx\") pod \"6135f634-50ec-4fc2-a539-354ea4d4fefe\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") "
Jul 02 20:05:15.641590 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.641452    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6135f634-50ec-4fc2-a539-354ea4d4fefe-extensions-socket-volume\") pod \"6135f634-50ec-4fc2-a539-354ea4d4fefe\" (UID: \"6135f634-50ec-4fc2-a539-354ea4d4fefe\") "
Jul 02 20:05:15.641839 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.641817    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6135f634-50ec-4fc2-a539-354ea4d4fefe-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6135f634-50ec-4fc2-a539-354ea4d4fefe" (UID: "6135f634-50ec-4fc2-a539-354ea4d4fefe"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:05:15.643643 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.643618    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6135f634-50ec-4fc2-a539-354ea4d4fefe-kube-api-access-qm7rx" (OuterVolumeSpecName: "kube-api-access-qm7rx") pod "6135f634-50ec-4fc2-a539-354ea4d4fefe" (UID: "6135f634-50ec-4fc2-a539-354ea4d4fefe"). InnerVolumeSpecName "kube-api-access-qm7rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:05:15.742139 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.742099    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm7rx\" (UniqueName: \"kubernetes.io/projected/6135f634-50ec-4fc2-a539-354ea4d4fefe-kube-api-access-qm7rx\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:05:15.742139 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:15.742131    2536 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6135f634-50ec-4fc2-a539-354ea4d4fefe-extensions-socket-volume\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:05:16.435024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:16.434995    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt" event={"ID":"6135f634-50ec-4fc2-a539-354ea4d4fefe","Type":"ContainerDied","Data":"2cd5b45261e664708b414b067d5f37be566ee3844d05df23a2c6b2bd2eb8494f"}
Jul 02 20:05:16.435412 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:16.435033    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"
Jul 02 20:05:16.435412 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:16.435063    2536 scope.go:117] "RemoveContainer" containerID="3b9ce02b228e07a9849622df4c12f40760fb97fb6aa38e1ec7d0a33201158af1"
Jul 02 20:05:16.453691 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:16.453667    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"]
Jul 02 20:05:16.461731 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:16.461712    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-llnkt"]
Jul 02 20:05:18.369085 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:18.369054    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6135f634-50ec-4fc2-a539-354ea4d4fefe" path="/var/lib/kubelet/pods/6135f634-50ec-4fc2-a539-354ea4d4fefe/volumes"
Jul 02 20:05:31.715943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.715911    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"]
Jul 02 20:05:31.716331 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.716223    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6135f634-50ec-4fc2-a539-354ea4d4fefe" containerName="manager"
Jul 02 20:05:31.716331 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.716234    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6135f634-50ec-4fc2-a539-354ea4d4fefe" containerName="manager"
Jul 02 20:05:31.716331 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.716312    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6135f634-50ec-4fc2-a539-354ea4d4fefe" containerName="manager"
Jul 02 20:05:31.720519 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.720497    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.722960 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.722944    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-twx78\""
Jul 02 20:05:31.730717 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.730697    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"]
Jul 02 20:05:31.868398 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868366    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868408    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868432    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868458    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868479    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/f0dc4238-60b0-4e2f-bb70-2045b337e151-kube-api-access-nc22b\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868497    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0dc4238-60b0-4e2f-bb70-2045b337e151-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868524    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868545    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.868819 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.868570    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.969868 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.969795    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.969868 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.969841    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.969868 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.969859    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970081 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.969881    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970081 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.969987    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/f0dc4238-60b0-4e2f-bb70-2045b337e151-kube-api-access-nc22b\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970081 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970024    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0dc4238-60b0-4e2f-bb70-2045b337e151-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970081 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970058    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970227    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970230    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970425 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970329    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970526    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970558    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970638    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.970834 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.970812    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0dc4238-60b0-4e2f-bb70-2045b337e151-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.972388 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.972368    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.972490 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.972464    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.977922 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.977902    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0dc4238-60b0-4e2f-bb70-2045b337e151-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:31.978173 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:31.978156    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/f0dc4238-60b0-4e2f-bb70-2045b337e151-kube-api-access-nc22b\") pod \"maas-default-gateway-openshift-default-56654dcf7-ph4nv\" (UID: \"f0dc4238-60b0-4e2f-bb70-2045b337e151\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:32.030895 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.030866    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:32.162807 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.162784    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"]
Jul 02 20:05:32.164704 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:05:32.164678    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0dc4238_60b0_4e2f_bb70_2045b337e151.slice/crio-71d196e25581cf06c7efff330bc07cd2e36e7baa2a096f74260f1c5921171afe WatchSource:0}: Error finding container 71d196e25581cf06c7efff330bc07cd2e36e7baa2a096f74260f1c5921171afe: Status 404 returned error can't find the container with id 71d196e25581cf06c7efff330bc07cd2e36e7baa2a096f74260f1c5921171afe
Jul 02 20:05:32.166905 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.166875    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:05:32.166975 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.166938    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:05:32.167015 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.166972    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:05:32.501333 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.501234    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" event={"ID":"f0dc4238-60b0-4e2f-bb70-2045b337e151","Type":"ContainerStarted","Data":"2b3e13727f3ea1c50752f0574c5af0bb9b844b8db550f393b288002ffd5814dc"}
Jul 02 20:05:32.501333 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.501308    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" event={"ID":"f0dc4238-60b0-4e2f-bb70-2045b337e151","Type":"ContainerStarted","Data":"71d196e25581cf06c7efff330bc07cd2e36e7baa2a096f74260f1c5921171afe"}
Jul 02 20:05:32.520435 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:32.520388    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" podStartSLOduration=1.520372804 podStartE2EDuration="1.520372804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:05:32.519741219 +0000 UTC m=+544.667509997" watchObservedRunningTime="2026-07-02 20:05:32.520372804 +0000 UTC m=+544.668141582"
Jul 02 20:05:33.031807 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:33.031772    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:33.036164 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:33.036141    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:33.506122 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:33.506099    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:33.506953 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:33.506931    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:05:34.049038 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.049008    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"]
Jul 02 20:05:34.052393 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.052374    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.054767 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.054749    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-km2p4\""
Jul 02 20:05:34.054767 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.054760    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 02 20:05:34.062538 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.062521    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"]
Jul 02 20:05:34.138758 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.138727    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"]
Jul 02 20:05:34.190867 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.190834    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch6cd\" (UniqueName: \"kubernetes.io/projected/dd35da70-9b72-4028-a4bc-2a33f19e796b-kube-api-access-ch6cd\") pod \"limitador-limitador-64c94fd64f-5xt4m\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.191024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.190877    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd35da70-9b72-4028-a4bc-2a33f19e796b-config-file\") pod \"limitador-limitador-64c94fd64f-5xt4m\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.292130 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.292098    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd35da70-9b72-4028-a4bc-2a33f19e796b-config-file\") pod \"limitador-limitador-64c94fd64f-5xt4m\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.292357 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.292168    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch6cd\" (UniqueName: \"kubernetes.io/projected/dd35da70-9b72-4028-a4bc-2a33f19e796b-kube-api-access-ch6cd\") pod \"limitador-limitador-64c94fd64f-5xt4m\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.292798 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.292773    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd35da70-9b72-4028-a4bc-2a33f19e796b-config-file\") pod \"limitador-limitador-64c94fd64f-5xt4m\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.310536 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.310479    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch6cd\" (UniqueName: \"kubernetes.io/projected/dd35da70-9b72-4028-a4bc-2a33f19e796b-kube-api-access-ch6cd\") pod \"limitador-limitador-64c94fd64f-5xt4m\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.363422 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.363393    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:34.490314 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.490273    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"]
Jul 02 20:05:34.492317 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:05:34.492273    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd35da70_9b72_4028_a4bc_2a33f19e796b.slice/crio-d3ebe0f9ae18e9395c6215a01497eb8a7377250e9dfcca27883963c70f6043c5 WatchSource:0}: Error finding container d3ebe0f9ae18e9395c6215a01497eb8a7377250e9dfcca27883963c70f6043c5: Status 404 returned error can't find the container with id d3ebe0f9ae18e9395c6215a01497eb8a7377250e9dfcca27883963c70f6043c5
Jul 02 20:05:34.511537 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.511512    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m" event={"ID":"dd35da70-9b72-4028-a4bc-2a33f19e796b","Type":"ContainerStarted","Data":"d3ebe0f9ae18e9395c6215a01497eb8a7377250e9dfcca27883963c70f6043c5"}
Jul 02 20:05:34.851481 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.851452    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-lspzw"]
Jul 02 20:05:34.855939 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.855917    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lspzw"
Jul 02 20:05:34.858425 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.858403    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q8bsr\""
Jul 02 20:05:34.861679 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.861658    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-lspzw"]
Jul 02 20:05:34.997198 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:34.997165    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpdzw\" (UniqueName: \"kubernetes.io/projected/c0f69cc9-705a-469a-8037-08be2edb2963-kube-api-access-qpdzw\") pod \"authorino-58cd955675-lspzw\" (UID: \"c0f69cc9-705a-469a-8037-08be2edb2963\") " pod="kuadrant-system/authorino-58cd955675-lspzw"
Jul 02 20:05:35.098149 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:35.098099    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpdzw\" (UniqueName: \"kubernetes.io/projected/c0f69cc9-705a-469a-8037-08be2edb2963-kube-api-access-qpdzw\") pod \"authorino-58cd955675-lspzw\" (UID: \"c0f69cc9-705a-469a-8037-08be2edb2963\") " pod="kuadrant-system/authorino-58cd955675-lspzw"
Jul 02 20:05:35.107313 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:35.107229    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpdzw\" (UniqueName: \"kubernetes.io/projected/c0f69cc9-705a-469a-8037-08be2edb2963-kube-api-access-qpdzw\") pod \"authorino-58cd955675-lspzw\" (UID: \"c0f69cc9-705a-469a-8037-08be2edb2963\") " pod="kuadrant-system/authorino-58cd955675-lspzw"
Jul 02 20:05:35.167126 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:35.167093    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lspzw"
Jul 02 20:05:35.345554 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:35.345517    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-lspzw"]
Jul 02 20:05:35.517902 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:35.517841    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lspzw" event={"ID":"c0f69cc9-705a-469a-8037-08be2edb2963","Type":"ContainerStarted","Data":"1f70b0f3d64c55246a686acabb4bd1dbb0c2336ac7e6db8dd5eb03683bbbe3ef"}
Jul 02 20:05:37.531649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:37.531431    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m" event={"ID":"dd35da70-9b72-4028-a4bc-2a33f19e796b","Type":"ContainerStarted","Data":"bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e"}
Jul 02 20:05:37.532168 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:37.532144    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:37.550388 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:37.550348    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m" podStartSLOduration=0.818030994 podStartE2EDuration="3.550332704s" totalImagesPullingTime="2.73230171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:05:37.549759062 +0000 UTC m=+549.697527873" watchObservedRunningTime="2026-07-02 20:05:37.550332704 +0000 UTC m=+549.698101482"
Jul 02 20:05:39.437649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:39.437609    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-lspzw"]
Jul 02 20:05:39.541887 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:39.541854    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lspzw" event={"ID":"c0f69cc9-705a-469a-8037-08be2edb2963","Type":"ContainerStarted","Data":"845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225"}
Jul 02 20:05:39.558641 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:39.558558    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-lspzw" podStartSLOduration=1.618803065 podStartE2EDuration="5.558543786s" totalImagesPullingTime="3.939740721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:05:39.558280214 +0000 UTC m=+551.706049011" watchObservedRunningTime="2026-07-02 20:05:39.558543786 +0000 UTC m=+551.706312565"
Jul 02 20:05:40.546453 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:40.546418    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-lspzw" podUID="c0f69cc9-705a-469a-8037-08be2edb2963" containerName="authorino" containerID="cri-o://845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225" gracePeriod=30
Jul 02 20:05:40.793644 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:40.793620    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lspzw"
Jul 02 20:05:40.952620 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:40.952587    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpdzw\" (UniqueName: \"kubernetes.io/projected/c0f69cc9-705a-469a-8037-08be2edb2963-kube-api-access-qpdzw\") pod \"c0f69cc9-705a-469a-8037-08be2edb2963\" (UID: \"c0f69cc9-705a-469a-8037-08be2edb2963\") "
Jul 02 20:05:40.954704 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:40.954679    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f69cc9-705a-469a-8037-08be2edb2963-kube-api-access-qpdzw" (OuterVolumeSpecName: "kube-api-access-qpdzw") pod "c0f69cc9-705a-469a-8037-08be2edb2963" (UID: "c0f69cc9-705a-469a-8037-08be2edb2963"). InnerVolumeSpecName "kube-api-access-qpdzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:05:41.054007 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.053972    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpdzw\" (UniqueName: \"kubernetes.io/projected/c0f69cc9-705a-469a-8037-08be2edb2963-kube-api-access-qpdzw\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:05:41.551749 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.551716    2536 generic.go:356] "Generic (PLEG): container finished" podID="c0f69cc9-705a-469a-8037-08be2edb2963" containerID="845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225" exitCode=0
Jul 02 20:05:41.551749 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.551753    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lspzw"
Jul 02 20:05:41.552205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.551798    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lspzw" event={"ID":"c0f69cc9-705a-469a-8037-08be2edb2963","Type":"ContainerDied","Data":"845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225"}
Jul 02 20:05:41.552205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.551835    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lspzw" event={"ID":"c0f69cc9-705a-469a-8037-08be2edb2963","Type":"ContainerDied","Data":"1f70b0f3d64c55246a686acabb4bd1dbb0c2336ac7e6db8dd5eb03683bbbe3ef"}
Jul 02 20:05:41.552205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.551853    2536 scope.go:117] "RemoveContainer" containerID="845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225"
Jul 02 20:05:41.560411 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.560396    2536 scope.go:117] "RemoveContainer" containerID="845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225"
Jul 02 20:05:41.560641 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:05:41.560623    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225\": container with ID starting with 845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225 not found: ID does not exist" containerID="845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225"
Jul 02 20:05:41.560678 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.560651    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225"} err="failed to get container status \"845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225\": rpc error: code = NotFound desc = could not find container \"845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225\": container with ID starting with 845b80ec42d4f03a9bcfece86175920bc4358140507e698e1adad37a17f6d225 not found: ID does not exist"
Jul 02 20:05:41.573659 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.573636    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-lspzw"]
Jul 02 20:05:41.577174 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:41.577154    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-lspzw"]
Jul 02 20:05:42.368695 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:42.368665    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f69cc9-705a-469a-8037-08be2edb2963" path="/var/lib/kubelet/pods/c0f69cc9-705a-469a-8037-08be2edb2963/volumes"
Jul 02 20:05:49.454952 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:49.454919    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"]
Jul 02 20:05:49.455505 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:49.455131    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m" podUID="dd35da70-9b72-4028-a4bc-2a33f19e796b" containerName="limitador" containerID="cri-o://bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e" gracePeriod=30
Jul 02 20:05:49.455996 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:49.455820    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:50.009619 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.009592    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:50.021154 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.021065    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch6cd\" (UniqueName: \"kubernetes.io/projected/dd35da70-9b72-4028-a4bc-2a33f19e796b-kube-api-access-ch6cd\") pod \"dd35da70-9b72-4028-a4bc-2a33f19e796b\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") "
Jul 02 20:05:50.021154 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.021115    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd35da70-9b72-4028-a4bc-2a33f19e796b-config-file\") pod \"dd35da70-9b72-4028-a4bc-2a33f19e796b\" (UID: \"dd35da70-9b72-4028-a4bc-2a33f19e796b\") "
Jul 02 20:05:50.021526 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.021496    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd35da70-9b72-4028-a4bc-2a33f19e796b-config-file" (OuterVolumeSpecName: "config-file") pod "dd35da70-9b72-4028-a4bc-2a33f19e796b" (UID: "dd35da70-9b72-4028-a4bc-2a33f19e796b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:05:50.023385 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.023359    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd35da70-9b72-4028-a4bc-2a33f19e796b-kube-api-access-ch6cd" (OuterVolumeSpecName: "kube-api-access-ch6cd") pod "dd35da70-9b72-4028-a4bc-2a33f19e796b" (UID: "dd35da70-9b72-4028-a4bc-2a33f19e796b"). InnerVolumeSpecName "kube-api-access-ch6cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:05:50.122279 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.122245    2536 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dd35da70-9b72-4028-a4bc-2a33f19e796b-config-file\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:05:50.122279 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.122273    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch6cd\" (UniqueName: \"kubernetes.io/projected/dd35da70-9b72-4028-a4bc-2a33f19e796b-kube-api-access-ch6cd\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:05:50.593674 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.593643    2536 generic.go:356] "Generic (PLEG): container finished" podID="dd35da70-9b72-4028-a4bc-2a33f19e796b" containerID="bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e" exitCode=0
Jul 02 20:05:50.594059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.593686    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"
Jul 02 20:05:50.594059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.593725    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m" event={"ID":"dd35da70-9b72-4028-a4bc-2a33f19e796b","Type":"ContainerDied","Data":"bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e"}
Jul 02 20:05:50.594059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.593778    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-5xt4m" event={"ID":"dd35da70-9b72-4028-a4bc-2a33f19e796b","Type":"ContainerDied","Data":"d3ebe0f9ae18e9395c6215a01497eb8a7377250e9dfcca27883963c70f6043c5"}
Jul 02 20:05:50.594059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.593799    2536 scope.go:117] "RemoveContainer" containerID="bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e"
Jul 02 20:05:50.601737 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.601722    2536 scope.go:117] "RemoveContainer" containerID="bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e"
Jul 02 20:05:50.602012 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:05:50.601993    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e\": container with ID starting with bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e not found: ID does not exist" containerID="bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e"
Jul 02 20:05:50.602059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.602021    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e"} err="failed to get container status \"bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e\": rpc error: code = NotFound desc = could not find container \"bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e\": container with ID starting with bbecd03b2631f0ed542a4f00fa0a911a5ddca76b562417c351f4449b1730317e not found: ID does not exist"
Jul 02 20:05:50.611088 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.611068    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"]
Jul 02 20:05:50.614593 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:50.614575    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-5xt4m"]
Jul 02 20:05:52.369008 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:52.368976    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd35da70-9b72-4028-a4bc-2a33f19e796b" path="/var/lib/kubelet/pods/dd35da70-9b72-4028-a4bc-2a33f19e796b/volumes"
Jul 02 20:05:55.086535 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.086497    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-67g7s"]
Jul 02 20:05:55.086908 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.086802    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd35da70-9b72-4028-a4bc-2a33f19e796b" containerName="limitador"
Jul 02 20:05:55.086908 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.086812    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd35da70-9b72-4028-a4bc-2a33f19e796b" containerName="limitador"
Jul 02 20:05:55.086908 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.086823    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f69cc9-705a-469a-8037-08be2edb2963" containerName="authorino"
Jul 02 20:05:55.086908 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.086828    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f69cc9-705a-469a-8037-08be2edb2963" containerName="authorino"
Jul 02 20:05:55.086908 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.086876    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f69cc9-705a-469a-8037-08be2edb2963" containerName="authorino"
Jul 02 20:05:55.086908 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.086885    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd35da70-9b72-4028-a4bc-2a33f19e796b" containerName="limitador"
Jul 02 20:05:55.092383 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.092362    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.095765 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.095744    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 02 20:05:55.095866 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.095744    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2zdpn\""
Jul 02 20:05:55.102277 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.102256    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-67g7s"]
Jul 02 20:05:55.159402 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.159368    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bvxn\" (UniqueName: \"kubernetes.io/projected/5ce510fc-dfb7-49f0-828a-8f72f90bf212-kube-api-access-9bvxn\") pod \"postgres-69fb4594cf-67g7s\" (UID: \"5ce510fc-dfb7-49f0-828a-8f72f90bf212\") " pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.159543 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.159435    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5ce510fc-dfb7-49f0-828a-8f72f90bf212-data\") pod \"postgres-69fb4594cf-67g7s\" (UID: \"5ce510fc-dfb7-49f0-828a-8f72f90bf212\") " pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.260552 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.260517    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvxn\" (UniqueName: \"kubernetes.io/projected/5ce510fc-dfb7-49f0-828a-8f72f90bf212-kube-api-access-9bvxn\") pod \"postgres-69fb4594cf-67g7s\" (UID: \"5ce510fc-dfb7-49f0-828a-8f72f90bf212\") " pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.260731 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.260565    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5ce510fc-dfb7-49f0-828a-8f72f90bf212-data\") pod \"postgres-69fb4594cf-67g7s\" (UID: \"5ce510fc-dfb7-49f0-828a-8f72f90bf212\") " pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.260876 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.260858    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5ce510fc-dfb7-49f0-828a-8f72f90bf212-data\") pod \"postgres-69fb4594cf-67g7s\" (UID: \"5ce510fc-dfb7-49f0-828a-8f72f90bf212\") " pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.270182 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.270163    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bvxn\" (UniqueName: \"kubernetes.io/projected/5ce510fc-dfb7-49f0-828a-8f72f90bf212-kube-api-access-9bvxn\") pod \"postgres-69fb4594cf-67g7s\" (UID: \"5ce510fc-dfb7-49f0-828a-8f72f90bf212\") " pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.405010 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.404988    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:05:55.532716 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.532687    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-67g7s"]
Jul 02 20:05:55.616481 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:05:55.616451    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-67g7s" event={"ID":"5ce510fc-dfb7-49f0-828a-8f72f90bf212","Type":"ContainerStarted","Data":"f15f0e4ef98226dc6c54edb82580caf89b71ff2a3040ca4f85cac15c54507916"}
Jul 02 20:06:00.642196 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:00.642164    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-67g7s" event={"ID":"5ce510fc-dfb7-49f0-828a-8f72f90bf212","Type":"ContainerStarted","Data":"54be5dc033d2846c125c99bce3d41ece8f28fb744650295f5ea9d03c733abfee"}
Jul 02 20:06:00.642548 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:00.642280    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:06:00.659052 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:00.659007    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-67g7s" podStartSLOduration=0.66991873 podStartE2EDuration="5.658993557s" totalImagesPullingTime="4.989074827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:06:00.658017572 +0000 UTC m=+572.805786378" watchObservedRunningTime="2026-07-02 20:06:00.658993557 +0000 UTC m=+572.806762335"
Jul 02 20:06:06.675147 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:06.675122    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-67g7s"
Jul 02 20:06:12.851041 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.851006    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b58f7797c-zxntg"]
Jul 02 20:06:12.854543 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.854521    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:12.858233 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.858210    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-crchz\""
Jul 02 20:06:12.858374 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.858254    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 02 20:06:12.858374 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.858260    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 02 20:06:12.864003 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.863971    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b58f7797c-zxntg"]
Jul 02 20:06:12.896669 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.896643    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-webhook-cert\") pod \"maas-controller-b58f7797c-zxntg\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") " pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:12.896813 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.896691    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxdgc\" (UniqueName: \"kubernetes.io/projected/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-kube-api-access-lxdgc\") pod \"maas-controller-b58f7797c-zxntg\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") " pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:12.997062 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.997029    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-webhook-cert\") pod \"maas-controller-b58f7797c-zxntg\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") " pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:12.997224 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.997075    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxdgc\" (UniqueName: \"kubernetes.io/projected/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-kube-api-access-lxdgc\") pod \"maas-controller-b58f7797c-zxntg\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") " pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:12.999492 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:12.999475    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-webhook-cert\") pod \"maas-controller-b58f7797c-zxntg\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") " pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:13.009577 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.009557    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxdgc\" (UniqueName: \"kubernetes.io/projected/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-kube-api-access-lxdgc\") pod \"maas-controller-b58f7797c-zxntg\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") " pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:13.014169 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.014146    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77558d7458-h8qvs"]
Jul 02 20:06:13.017556 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.017538    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.027096 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.027075    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77558d7458-h8qvs"]
Jul 02 20:06:13.098132 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.098103    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2621df57-13e9-4afd-898f-1f9adcf61b1e-webhook-cert\") pod \"maas-controller-77558d7458-h8qvs\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") " pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.098132 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.098133    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx56g\" (UniqueName: \"kubernetes.io/projected/2621df57-13e9-4afd-898f-1f9adcf61b1e-kube-api-access-qx56g\") pod \"maas-controller-77558d7458-h8qvs\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") " pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.165149 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.165123    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:13.198930 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.198892    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2621df57-13e9-4afd-898f-1f9adcf61b1e-webhook-cert\") pod \"maas-controller-77558d7458-h8qvs\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") " pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.199052 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.198947    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx56g\" (UniqueName: \"kubernetes.io/projected/2621df57-13e9-4afd-898f-1f9adcf61b1e-kube-api-access-qx56g\") pod \"maas-controller-77558d7458-h8qvs\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") " pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.201444 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.201421    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2621df57-13e9-4afd-898f-1f9adcf61b1e-webhook-cert\") pod \"maas-controller-77558d7458-h8qvs\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") " pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.207627 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.207606    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx56g\" (UniqueName: \"kubernetes.io/projected/2621df57-13e9-4afd-898f-1f9adcf61b1e-kube-api-access-qx56g\") pod \"maas-controller-77558d7458-h8qvs\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") " pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.292901 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.292879    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b58f7797c-zxntg"]
Jul 02 20:06:13.295660 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:06:13.295633    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049dafa4_9ade_4485_a4fb_cfe9b8012cc1.slice/crio-80c3fdf466c5d4f57bb75b41def4ea6bda476a6147de681dc005c9cb73d8dc44 WatchSource:0}: Error finding container 80c3fdf466c5d4f57bb75b41def4ea6bda476a6147de681dc005c9cb73d8dc44: Status 404 returned error can't find the container with id 80c3fdf466c5d4f57bb75b41def4ea6bda476a6147de681dc005c9cb73d8dc44
Jul 02 20:06:13.327581 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.327562    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:13.456952 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.456884    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77558d7458-h8qvs"]
Jul 02 20:06:13.459070 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:06:13.459047    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2621df57_13e9_4afd_898f_1f9adcf61b1e.slice/crio-5796ee9413ff35eb5c113e37e080366d029ac2dd1245efb21315d7d8d4cb6ad1 WatchSource:0}: Error finding container 5796ee9413ff35eb5c113e37e080366d029ac2dd1245efb21315d7d8d4cb6ad1: Status 404 returned error can't find the container with id 5796ee9413ff35eb5c113e37e080366d029ac2dd1245efb21315d7d8d4cb6ad1
Jul 02 20:06:13.696603 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.696568    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77558d7458-h8qvs" event={"ID":"2621df57-13e9-4afd-898f-1f9adcf61b1e","Type":"ContainerStarted","Data":"5796ee9413ff35eb5c113e37e080366d029ac2dd1245efb21315d7d8d4cb6ad1"}
Jul 02 20:06:13.697605 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:13.697584    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f7797c-zxntg" event={"ID":"049dafa4-9ade-4485-a4fb-cfe9b8012cc1","Type":"ContainerStarted","Data":"80c3fdf466c5d4f57bb75b41def4ea6bda476a6147de681dc005c9cb73d8dc44"}
Jul 02 20:06:16.714371 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:16.714332    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f7797c-zxntg" event={"ID":"049dafa4-9ade-4485-a4fb-cfe9b8012cc1","Type":"ContainerStarted","Data":"03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa"}
Jul 02 20:06:16.714804 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:16.714399    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:16.715782 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:16.715760    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77558d7458-h8qvs" event={"ID":"2621df57-13e9-4afd-898f-1f9adcf61b1e","Type":"ContainerStarted","Data":"a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4"}
Jul 02 20:06:16.715889 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:16.715845    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:16.733059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:16.733016    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b58f7797c-zxntg" podStartSLOduration=1.9370741310000001 podStartE2EDuration="4.733002381s" totalImagesPullingTime="2.79592825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:06:16.73136594 +0000 UTC m=+588.879134720" watchObservedRunningTime="2026-07-02 20:06:16.733002381 +0000 UTC m=+588.880771162"
Jul 02 20:06:16.748692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:16.748654    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77558d7458-h8qvs" podStartSLOduration=2.125405418 podStartE2EDuration="4.748637651s" totalImagesPullingTime="2.623232233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:06:16.747660517 +0000 UTC m=+588.895429309" watchObservedRunningTime="2026-07-02 20:06:16.748637651 +0000 UTC m=+588.896406433"
Jul 02 20:06:27.725812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:27.725779    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:06:27.726247 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:27.725862    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:27.784025 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:27.783987    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b58f7797c-zxntg"]
Jul 02 20:06:27.784261 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:27.784237    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-b58f7797c-zxntg" podUID="049dafa4-9ade-4485-a4fb-cfe9b8012cc1" containerName="manager" containerID="cri-o://03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa" gracePeriod=30
Jul 02 20:06:28.027633 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.027608    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:28.116379 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.116354    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxdgc\" (UniqueName: \"kubernetes.io/projected/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-kube-api-access-lxdgc\") pod \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") "
Jul 02 20:06:28.116533 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.116419    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-webhook-cert\") pod \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\" (UID: \"049dafa4-9ade-4485-a4fb-cfe9b8012cc1\") "
Jul 02 20:06:28.118487 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.118458    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-kube-api-access-lxdgc" (OuterVolumeSpecName: "kube-api-access-lxdgc") pod "049dafa4-9ade-4485-a4fb-cfe9b8012cc1" (UID: "049dafa4-9ade-4485-a4fb-cfe9b8012cc1"). InnerVolumeSpecName "kube-api-access-lxdgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:06:28.118586 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.118487    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "049dafa4-9ade-4485-a4fb-cfe9b8012cc1" (UID: "049dafa4-9ade-4485-a4fb-cfe9b8012cc1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:06:28.216930 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.216906    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxdgc\" (UniqueName: \"kubernetes.io/projected/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-kube-api-access-lxdgc\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:06:28.216930 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.216928    2536 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/049dafa4-9ade-4485-a4fb-cfe9b8012cc1-webhook-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:06:28.775069 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.775034    2536 generic.go:356] "Generic (PLEG): container finished" podID="049dafa4-9ade-4485-a4fb-cfe9b8012cc1" containerID="03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa" exitCode=0
Jul 02 20:06:28.775499 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.775080    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b58f7797c-zxntg"
Jul 02 20:06:28.775499 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.775124    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f7797c-zxntg" event={"ID":"049dafa4-9ade-4485-a4fb-cfe9b8012cc1","Type":"ContainerDied","Data":"03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa"}
Jul 02 20:06:28.775499 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.775174    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b58f7797c-zxntg" event={"ID":"049dafa4-9ade-4485-a4fb-cfe9b8012cc1","Type":"ContainerDied","Data":"80c3fdf466c5d4f57bb75b41def4ea6bda476a6147de681dc005c9cb73d8dc44"}
Jul 02 20:06:28.775499 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.775198    2536 scope.go:117] "RemoveContainer" containerID="03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa"
Jul 02 20:06:28.783504 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.783459    2536 scope.go:117] "RemoveContainer" containerID="03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa"
Jul 02 20:06:28.783690 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:06:28.783675    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa\": container with ID starting with 03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa not found: ID does not exist" containerID="03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa"
Jul 02 20:06:28.783734 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.783696    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa"} err="failed to get container status \"03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa\": rpc error: code = NotFound desc = could not find container \"03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa\": container with ID starting with 03cbba9230186301f676d1ee8c16ac51718caad402376071b2b53cfe2096b1aa not found: ID does not exist"
Jul 02 20:06:28.791052 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.791029    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b58f7797c-zxntg"]
Jul 02 20:06:28.794842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:28.794824    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b58f7797c-zxntg"]
Jul 02 20:06:30.369114 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:30.369082    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049dafa4-9ade-4485-a4fb-cfe9b8012cc1" path="/var/lib/kubelet/pods/049dafa4-9ade-4485-a4fb-cfe9b8012cc1/volumes"
Jul 02 20:06:47.695398 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.695359    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6496b86fff-7dhvk"]
Jul 02 20:06:47.695854 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.695689    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="049dafa4-9ade-4485-a4fb-cfe9b8012cc1" containerName="manager"
Jul 02 20:06:47.695854 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.695700    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="049dafa4-9ade-4485-a4fb-cfe9b8012cc1" containerName="manager"
Jul 02 20:06:47.695854 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.695757    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="049dafa4-9ade-4485-a4fb-cfe9b8012cc1" containerName="manager"
Jul 02 20:06:47.718334 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.718310    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6496b86fff-7dhvk"]
Jul 02 20:06:47.718499 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.718435    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.724461 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.724437    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-cwtmg\""
Jul 02 20:06:47.724588 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.724528    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 02 20:06:47.879995 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.879967    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nzjl\" (UniqueName: \"kubernetes.io/projected/2a36a87b-3ba5-4f97-809f-0c48c20673bd-kube-api-access-5nzjl\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.880164 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.880024    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36a87b-3ba5-4f97-809f-0c48c20673bd-openshift-service-ca\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.880164 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.880064    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a36a87b-3ba5-4f97-809f-0c48c20673bd-maas-api-tls\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.980552 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.980484    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36a87b-3ba5-4f97-809f-0c48c20673bd-openshift-service-ca\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.980552 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.980521    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a36a87b-3ba5-4f97-809f-0c48c20673bd-maas-api-tls\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.980720 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.980573    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nzjl\" (UniqueName: \"kubernetes.io/projected/2a36a87b-3ba5-4f97-809f-0c48c20673bd-kube-api-access-5nzjl\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.980720 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:06:47.980689    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 02 20:06:47.980790 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:06:47.980764    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a36a87b-3ba5-4f97-809f-0c48c20673bd-maas-api-tls podName:2a36a87b-3ba5-4f97-809f-0c48c20673bd nodeName:}" failed. No retries permitted until 2026-07-02 20:06:48.48074316 +0000 UTC m=+620.628511919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a36a87b-3ba5-4f97-809f-0c48c20673bd-maas-api-tls") pod "maas-api-6496b86fff-7dhvk" (UID: "2a36a87b-3ba5-4f97-809f-0c48c20673bd") : secret "maas-api-serving-cert" not found
Jul 02 20:06:47.981172 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.981154    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a36a87b-3ba5-4f97-809f-0c48c20673bd-openshift-service-ca\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:47.992734 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:47.992713    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nzjl\" (UniqueName: \"kubernetes.io/projected/2a36a87b-3ba5-4f97-809f-0c48c20673bd-kube-api-access-5nzjl\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:48.245926 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.245854    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-plmr7"]
Jul 02 20:06:48.302165 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.302133    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-plmr7"]
Jul 02 20:06:48.302343 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.302174    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bjwcc"]
Jul 02 20:06:48.302343 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.302249    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.304687 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.304652    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m2pnp\""
Jul 02 20:06:48.304789 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.304764    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 02 20:06:48.317495 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.317473    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bjwcc"]
Jul 02 20:06:48.317615 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.317566    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.384182 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.384146    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptxt4\" (UniqueName: \"kubernetes.io/projected/8de0d3e2-25e5-44bf-be59-15196ad11db1-kube-api-access-ptxt4\") pod \"payload-pre-processing-7964457696-plmr7\" (UID: \"8de0d3e2-25e5-44bf-be59-15196ad11db1\") " pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.384365 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.384202    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8de0d3e2-25e5-44bf-be59-15196ad11db1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-plmr7\" (UID: \"8de0d3e2-25e5-44bf-be59-15196ad11db1\") " pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.484943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.484907    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd715f0-aa66-4d09-bf19-b634fa42b163-plugins-config-volume\") pod \"payload-processing-58f86c7798-bjwcc\" (UID: \"0dd715f0-aa66-4d09-bf19-b634fa42b163\") " pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.484943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.484941    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ftrq\" (UniqueName: \"kubernetes.io/projected/0dd715f0-aa66-4d09-bf19-b634fa42b163-kube-api-access-8ftrq\") pod \"payload-processing-58f86c7798-bjwcc\" (UID: \"0dd715f0-aa66-4d09-bf19-b634fa42b163\") " pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.485185 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.484971    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a36a87b-3ba5-4f97-809f-0c48c20673bd-maas-api-tls\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:48.485185 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.484992    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptxt4\" (UniqueName: \"kubernetes.io/projected/8de0d3e2-25e5-44bf-be59-15196ad11db1-kube-api-access-ptxt4\") pod \"payload-pre-processing-7964457696-plmr7\" (UID: \"8de0d3e2-25e5-44bf-be59-15196ad11db1\") " pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.485185 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.485069    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8de0d3e2-25e5-44bf-be59-15196ad11db1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-plmr7\" (UID: \"8de0d3e2-25e5-44bf-be59-15196ad11db1\") " pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.485851 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.485832    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8de0d3e2-25e5-44bf-be59-15196ad11db1-plugins-config-volume\") pod \"payload-pre-processing-7964457696-plmr7\" (UID: \"8de0d3e2-25e5-44bf-be59-15196ad11db1\") " pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.487506 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.487485    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a36a87b-3ba5-4f97-809f-0c48c20673bd-maas-api-tls\") pod \"maas-api-6496b86fff-7dhvk\" (UID: \"2a36a87b-3ba5-4f97-809f-0c48c20673bd\") " pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:48.492941 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.492919    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptxt4\" (UniqueName: \"kubernetes.io/projected/8de0d3e2-25e5-44bf-be59-15196ad11db1-kube-api-access-ptxt4\") pod \"payload-pre-processing-7964457696-plmr7\" (UID: \"8de0d3e2-25e5-44bf-be59-15196ad11db1\") " pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.586493 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.586399    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd715f0-aa66-4d09-bf19-b634fa42b163-plugins-config-volume\") pod \"payload-processing-58f86c7798-bjwcc\" (UID: \"0dd715f0-aa66-4d09-bf19-b634fa42b163\") " pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.586493 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.586457    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ftrq\" (UniqueName: \"kubernetes.io/projected/0dd715f0-aa66-4d09-bf19-b634fa42b163-kube-api-access-8ftrq\") pod \"payload-processing-58f86c7798-bjwcc\" (UID: \"0dd715f0-aa66-4d09-bf19-b634fa42b163\") " pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.587034 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.587017    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0dd715f0-aa66-4d09-bf19-b634fa42b163-plugins-config-volume\") pod \"payload-processing-58f86c7798-bjwcc\" (UID: \"0dd715f0-aa66-4d09-bf19-b634fa42b163\") " pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.594607 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.594589    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ftrq\" (UniqueName: \"kubernetes.io/projected/0dd715f0-aa66-4d09-bf19-b634fa42b163-kube-api-access-8ftrq\") pod \"payload-processing-58f86c7798-bjwcc\" (UID: \"0dd715f0-aa66-4d09-bf19-b634fa42b163\") " pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.612123 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.612104    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:48.626894 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.626872    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:48.628931 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.628770    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:48.771674 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.771648    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-plmr7"]
Jul 02 20:06:48.775316 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:06:48.775268    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de0d3e2_25e5_44bf_be59_15196ad11db1.slice/crio-e89ea457408c5a82e502cb20fd006759a1d5baab02b181cdb5141c70222f1e5b WatchSource:0}: Error finding container e89ea457408c5a82e502cb20fd006759a1d5baab02b181cdb5141c70222f1e5b: Status 404 returned error can't find the container with id e89ea457408c5a82e502cb20fd006759a1d5baab02b181cdb5141c70222f1e5b
Jul 02 20:06:48.793631 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.793128    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bjwcc"]
Jul 02 20:06:48.795096 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:06:48.795049    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd715f0_aa66_4d09_bf19_b634fa42b163.slice/crio-de2202bf18ae08b95cf60fc1e73e4e44b19dbcb11e21ea33364fb8daa95c5999 WatchSource:0}: Error finding container de2202bf18ae08b95cf60fc1e73e4e44b19dbcb11e21ea33364fb8daa95c5999: Status 404 returned error can't find the container with id de2202bf18ae08b95cf60fc1e73e4e44b19dbcb11e21ea33364fb8daa95c5999
Jul 02 20:06:48.818910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.818887    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6496b86fff-7dhvk"]
Jul 02 20:06:48.821415 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:06:48.821393    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a36a87b_3ba5_4f97_809f_0c48c20673bd.slice/crio-53380139ba35d7bf6a0303c6112dce908148a3eee701c26c168bf4978719063a WatchSource:0}: Error finding container 53380139ba35d7bf6a0303c6112dce908148a3eee701c26c168bf4978719063a: Status 404 returned error can't find the container with id 53380139ba35d7bf6a0303c6112dce908148a3eee701c26c168bf4978719063a
Jul 02 20:06:48.858647 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.858550    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bjwcc" event={"ID":"0dd715f0-aa66-4d09-bf19-b634fa42b163","Type":"ContainerStarted","Data":"de2202bf18ae08b95cf60fc1e73e4e44b19dbcb11e21ea33364fb8daa95c5999"}
Jul 02 20:06:48.859850 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.859821    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-plmr7" event={"ID":"8de0d3e2-25e5-44bf-be59-15196ad11db1","Type":"ContainerStarted","Data":"e89ea457408c5a82e502cb20fd006759a1d5baab02b181cdb5141c70222f1e5b"}
Jul 02 20:06:48.861038 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:48.861017    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6496b86fff-7dhvk" event={"ID":"2a36a87b-3ba5-4f97-809f-0c48c20673bd","Type":"ContainerStarted","Data":"53380139ba35d7bf6a0303c6112dce908148a3eee701c26c168bf4978719063a"}
Jul 02 20:06:51.893974 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.893942    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bjwcc" event={"ID":"0dd715f0-aa66-4d09-bf19-b634fa42b163","Type":"ContainerStarted","Data":"170ed1c2589e65426d7ea05c37834871248f31488877dd7bbe694a900eb5e718"}
Jul 02 20:06:51.894361 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.894051    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:06:51.895961 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.895939    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-plmr7" event={"ID":"8de0d3e2-25e5-44bf-be59-15196ad11db1","Type":"ContainerStarted","Data":"c8d72ec155b978089d723aa83ae18aefc5369e43195edf34b1762c988255becd"}
Jul 02 20:06:51.896121 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.896087    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:06:51.897526 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.897450    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6496b86fff-7dhvk" event={"ID":"2a36a87b-3ba5-4f97-809f-0c48c20673bd","Type":"ContainerStarted","Data":"9b306906bae4eecc3c588a3d015dcd34d51af61dd6e251814ba945a131cf9b88"}
Jul 02 20:06:51.897526 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.897494    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:51.924643 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.924600    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-bjwcc" podStartSLOduration=1.775834651 podStartE2EDuration="3.924588449s" totalImagesPullingTime="2.148753798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:06:51.924317197 +0000 UTC m=+624.072085975" watchObservedRunningTime="2026-07-02 20:06:51.924588449 +0000 UTC m=+624.072357244"
Jul 02 20:06:51.941096 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.941060    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-plmr7" podStartSLOduration=1.776688698 podStartE2EDuration="3.941038352s" totalImagesPullingTime="2.164349654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:06:51.940255076 +0000 UTC m=+624.088023869" watchObservedRunningTime="2026-07-02 20:06:51.941038352 +0000 UTC m=+624.088807130"
Jul 02 20:06:51.964308 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:51.960342    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6496b86fff-7dhvk" podStartSLOduration=2.840434827 podStartE2EDuration="4.960324925s" totalImagesPullingTime="2.119890098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:06:51.959248262 +0000 UTC m=+624.107017056" watchObservedRunningTime="2026-07-02 20:06:51.960324925 +0000 UTC m=+624.108093705"
Jul 02 20:06:55.917437 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:55.917362    2536 generic.go:356] "Generic (PLEG): container finished" podID="f0dc4238-60b0-4e2f-bb70-2045b337e151" containerID="2b3e13727f3ea1c50752f0574c5af0bb9b844b8db550f393b288002ffd5814dc" exitCode=0
Jul 02 20:06:55.917437 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:55.917403    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" event={"ID":"f0dc4238-60b0-4e2f-bb70-2045b337e151","Type":"ContainerDied","Data":"2b3e13727f3ea1c50752f0574c5af0bb9b844b8db550f393b288002ffd5814dc"}
Jul 02 20:06:55.917833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:55.917789    2536 scope.go:117] "RemoveContainer" containerID="2b3e13727f3ea1c50752f0574c5af0bb9b844b8db550f393b288002ffd5814dc"
Jul 02 20:06:55.918790 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:55.918767    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:06:55.918883 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:55.918835    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:06:55.918883 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:55.918868    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:06:56.923721 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:56.923672    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" event={"ID":"f0dc4238-60b0-4e2f-bb70-2045b337e151","Type":"ContainerStarted","Data":"00dd6fb67f24d23b4cd6408890d629c1dd424895d91585a6044e825c44343838"}
Jul 02 20:06:57.031596 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:57.031567    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:06:57.031781 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:57.031759    2536 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ph4nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 02 20:06:57.031832 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:57.031806    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" podUID="f0dc4238-60b0-4e2f-bb70-2045b337e151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 02 20:06:57.908259 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:57.908228    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6496b86fff-7dhvk"
Jul 02 20:06:58.031808 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:58.031776    2536 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ph4nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 02 20:06:58.032175 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:58.031830    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" podUID="f0dc4238-60b0-4e2f-bb70-2045b337e151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 02 20:06:59.031950 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:59.031915    2536 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ph4nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 02 20:06:59.032358 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:06:59.031999    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" podUID="f0dc4238-60b0-4e2f-bb70-2045b337e151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 02 20:07:00.031496 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:00.031464    2536 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ph4nv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 02 20:07:00.031658 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:00.031522    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv" podUID="f0dc4238-60b0-4e2f-bb70-2045b337e151" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 02 20:07:01.035873 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:01.035845    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:07:01.036334 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:01.036309    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:07:01.037174 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:01.037152    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ph4nv"
Jul 02 20:07:02.903955 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:02.903927    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-bjwcc"
Jul 02 20:07:02.904459 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:02.904062    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-plmr7"
Jul 02 20:07:36.100535 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.100501    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"]
Jul 02 20:07:36.104783 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.104763    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.108105 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.108080    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 02 20:07:36.108226 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.108200    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-d2n2k\""
Jul 02 20:07:36.108314 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.108269    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 02 20:07:36.108382 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.108364    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 02 20:07:36.113098 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.113076    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"]
Jul 02 20:07:36.165059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.165027    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.165191 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.165071    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.165191 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.165104    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjl4\" (UniqueName: \"kubernetes.io/projected/b969f0ae-f0c9-4666-a804-25ff3429388a-kube-api-access-5wjl4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.165191 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.165166    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969f0ae-f0c9-4666-a804-25ff3429388a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.165331 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.165199    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.165331 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.165246    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266267 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266234    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266267 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266270    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266314    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjl4\" (UniqueName: \"kubernetes.io/projected/b969f0ae-f0c9-4666-a804-25ff3429388a-kube-api-access-5wjl4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266355    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969f0ae-f0c9-4666-a804-25ff3429388a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266375    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266401    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266764 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266734    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266897 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266784    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.266897 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.266799    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.268684 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.268661    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b969f0ae-f0c9-4666-a804-25ff3429388a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.268807 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.268792    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b969f0ae-f0c9-4666-a804-25ff3429388a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.275573 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.275554    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjl4\" (UniqueName: \"kubernetes.io/projected/b969f0ae-f0c9-4666-a804-25ff3429388a-kube-api-access-5wjl4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mprf\" (UID: \"b969f0ae-f0c9-4666-a804-25ff3429388a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.414943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.414920    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:36.566310 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.566269    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"]
Jul 02 20:07:36.567802 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:07:36.567778    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb969f0ae_f0c9_4666_a804_25ff3429388a.slice/crio-d3b72f49f07febc106a90038b53a38a2e20799e54996dec6e93fad0deba46847 WatchSource:0}: Error finding container d3b72f49f07febc106a90038b53a38a2e20799e54996dec6e93fad0deba46847: Status 404 returned error can't find the container with id d3b72f49f07febc106a90038b53a38a2e20799e54996dec6e93fad0deba46847
Jul 02 20:07:36.569068 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:36.569049    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 20:07:37.106098 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:37.106017    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf" event={"ID":"b969f0ae-f0c9-4666-a804-25ff3429388a","Type":"ContainerStarted","Data":"d3b72f49f07febc106a90038b53a38a2e20799e54996dec6e93fad0deba46847"}
Jul 02 20:07:38.111901 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:38.111830    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf" event={"ID":"b969f0ae-f0c9-4666-a804-25ff3429388a","Type":"ContainerStarted","Data":"a786ba446245bf190dd9d6221325c4554cfde5c4f22388fc544da33f93276e9e"}
Jul 02 20:07:38.128960 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:38.128909    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf" podStartSLOduration=1.055092323 podStartE2EDuration="2.12889267s" totalImagesPullingTime="1.073800347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:07:38.127266189 +0000 UTC m=+670.275034969" watchObservedRunningTime="2026-07-02 20:07:38.12889267 +0000 UTC m=+670.276661452"
Jul 02 20:07:46.415074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:46.415044    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:46.415611 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:46.415097    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:46.427138 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:46.427109    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:07:47.164939 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:07:47.164911    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mprf"
Jul 02 20:08:12.610190 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.610158    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"]
Jul 02 20:08:12.616024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.616002    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.618694 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.618676    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 02 20:08:12.624970 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.624938    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"]
Jul 02 20:08:12.778477 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.778442    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.778477 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.778478    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.778810 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.778495    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.778810 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.778535    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.778810 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.778624    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n854\" (UniqueName: \"kubernetes.io/projected/91baca46-756a-417c-8a29-e95cc9de74c5-kube-api-access-5n854\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.778810 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.778670    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91baca46-756a-417c-8a29-e95cc9de74c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880069 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880046    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91baca46-756a-417c-8a29-e95cc9de74c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880236 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880095    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880236 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880118    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880236 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880139    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880236 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880198    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880447 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880348    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n854\" (UniqueName: \"kubernetes.io/projected/91baca46-756a-417c-8a29-e95cc9de74c5-kube-api-access-5n854\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880660    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880665    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.880812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.880728    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.882482 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.882456    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91baca46-756a-417c-8a29-e95cc9de74c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.882754 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.882737    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91baca46-756a-417c-8a29-e95cc9de74c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.889260 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.889238    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n854\" (UniqueName: \"kubernetes.io/projected/91baca46-756a-417c-8a29-e95cc9de74c5-kube-api-access-5n854\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p\" (UID: \"91baca46-756a-417c-8a29-e95cc9de74c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:12.928436 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:12.928413    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:13.061691 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:13.061667    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"]
Jul 02 20:08:13.063777 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:08:13.063737    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91baca46_756a_417c_8a29_e95cc9de74c5.slice/crio-f7fea8813b7158ca6e972ff550920d051166f6d1f70e0b66a9a2bd6eb80302b3 WatchSource:0}: Error finding container f7fea8813b7158ca6e972ff550920d051166f6d1f70e0b66a9a2bd6eb80302b3: Status 404 returned error can't find the container with id f7fea8813b7158ca6e972ff550920d051166f6d1f70e0b66a9a2bd6eb80302b3
Jul 02 20:08:13.275071 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:13.274989    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p" event={"ID":"91baca46-756a-417c-8a29-e95cc9de74c5","Type":"ContainerStarted","Data":"f7fea8813b7158ca6e972ff550920d051166f6d1f70e0b66a9a2bd6eb80302b3"}
Jul 02 20:08:14.280472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:14.280438    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p" event={"ID":"91baca46-756a-417c-8a29-e95cc9de74c5","Type":"ContainerStarted","Data":"8a8f584ec62acf386f39a1b8f4d85f29b5d600d9664ae0be23677fc062d81847"}
Jul 02 20:08:14.299350 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:14.299305    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p" podStartSLOduration=2.085975531 podStartE2EDuration="2.299266571s" totalImagesPullingTime="213.29104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:08:14.297911819 +0000 UTC m=+706.445680618" watchObservedRunningTime="2026-07-02 20:08:14.299266571 +0000 UTC m=+706.447035353"
Jul 02 20:08:22.929104 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:22.929020    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:22.929104 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:22.929081    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:22.941244 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:22.941223    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:22.996943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:22.996914    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"]
Jul 02 20:08:23.000608 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.000585    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.003501 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.003481    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 02 20:08:23.011966 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.011941    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"]
Jul 02 20:08:23.060983 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.060950    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.061140 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.060984    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.061140 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.061009    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.061140 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.061093    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjs2d\" (UniqueName: \"kubernetes.io/projected/3883fa0d-efd2-4805-bb26-957fe4591ea5-kube-api-access-xjs2d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.061280 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.061186    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3883fa0d-efd2-4805-bb26-957fe4591ea5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.061280 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.061202    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.161715 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.161680    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.161894 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.161723    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.161894 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.161741    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.161894 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.161779    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjs2d\" (UniqueName: \"kubernetes.io/projected/3883fa0d-efd2-4805-bb26-957fe4591ea5-kube-api-access-xjs2d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.161894 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.161877    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3883fa0d-efd2-4805-bb26-957fe4591ea5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.162145 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.161906    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.162206 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.162184    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.162413 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.162360    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.162413 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.162381    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.164156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.164135    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3883fa0d-efd2-4805-bb26-957fe4591ea5-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.164620 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.164602    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3883fa0d-efd2-4805-bb26-957fe4591ea5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.170978 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.170957    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjs2d\" (UniqueName: \"kubernetes.io/projected/3883fa0d-efd2-4805-bb26-957fe4591ea5-kube-api-access-xjs2d\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j\" (UID: \"3883fa0d-efd2-4805-bb26-957fe4591ea5\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.310454 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.310373    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:23.333441 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.333420    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p"
Jul 02 20:08:23.457173 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:23.457148    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"]
Jul 02 20:08:23.459676 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:08:23.459653    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3883fa0d_efd2_4805_bb26_957fe4591ea5.slice/crio-ebcb0fae3136a99a9efbfbb3940b200819bdd85024c4f892a851f2daa62b398f WatchSource:0}: Error finding container ebcb0fae3136a99a9efbfbb3940b200819bdd85024c4f892a851f2daa62b398f: Status 404 returned error can't find the container with id ebcb0fae3136a99a9efbfbb3940b200819bdd85024c4f892a851f2daa62b398f
Jul 02 20:08:24.325253 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:24.325216    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j" event={"ID":"3883fa0d-efd2-4805-bb26-957fe4591ea5","Type":"ContainerStarted","Data":"6bed74bacd77903bee7947d47e03d8b030fbb563d58744f4293eaacc6c0f87e2"}
Jul 02 20:08:24.325652 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:24.325262    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j" event={"ID":"3883fa0d-efd2-4805-bb26-957fe4591ea5","Type":"ContainerStarted","Data":"ebcb0fae3136a99a9efbfbb3940b200819bdd85024c4f892a851f2daa62b398f"}
Jul 02 20:08:24.343774 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:24.343728    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j" podStartSLOduration=2.069603787 podStartE2EDuration="2.343711637s" totalImagesPullingTime="274.10785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:08:24.341935155 +0000 UTC m=+716.489703925" watchObservedRunningTime="2026-07-02 20:08:24.343711637 +0000 UTC m=+716.491480416"
Jul 02 20:08:33.311045 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:33.311013    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:33.311450 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:33.311061    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:33.323567 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:33.323544    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:08:33.377981 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:08:33.377947    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j"
Jul 02 20:10:32.115462 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.115429    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77558d7458-h8qvs"]
Jul 02 20:10:32.115902 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.115614    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-77558d7458-h8qvs" podUID="2621df57-13e9-4afd-898f-1f9adcf61b1e" containerName="manager" containerID="cri-o://a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4" gracePeriod=30
Jul 02 20:10:32.365835 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.365752    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:10:32.461353 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.461318    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx56g\" (UniqueName: \"kubernetes.io/projected/2621df57-13e9-4afd-898f-1f9adcf61b1e-kube-api-access-qx56g\") pod \"2621df57-13e9-4afd-898f-1f9adcf61b1e\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") "
Jul 02 20:10:32.461515 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.461421    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2621df57-13e9-4afd-898f-1f9adcf61b1e-webhook-cert\") pod \"2621df57-13e9-4afd-898f-1f9adcf61b1e\" (UID: \"2621df57-13e9-4afd-898f-1f9adcf61b1e\") "
Jul 02 20:10:32.463519 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.463486    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2621df57-13e9-4afd-898f-1f9adcf61b1e-kube-api-access-qx56g" (OuterVolumeSpecName: "kube-api-access-qx56g") pod "2621df57-13e9-4afd-898f-1f9adcf61b1e" (UID: "2621df57-13e9-4afd-898f-1f9adcf61b1e"). InnerVolumeSpecName "kube-api-access-qx56g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:10:32.463519 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.463504    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2621df57-13e9-4afd-898f-1f9adcf61b1e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2621df57-13e9-4afd-898f-1f9adcf61b1e" (UID: "2621df57-13e9-4afd-898f-1f9adcf61b1e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:10:32.562634 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.562601    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx56g\" (UniqueName: \"kubernetes.io/projected/2621df57-13e9-4afd-898f-1f9adcf61b1e-kube-api-access-qx56g\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:10:32.562634 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.562628    2536 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2621df57-13e9-4afd-898f-1f9adcf61b1e-webhook-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:10:32.908548 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.908519    2536 generic.go:356] "Generic (PLEG): container finished" podID="2621df57-13e9-4afd-898f-1f9adcf61b1e" containerID="a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4" exitCode=0
Jul 02 20:10:32.908741 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.908563    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77558d7458-h8qvs"
Jul 02 20:10:32.908741 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.908580    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77558d7458-h8qvs" event={"ID":"2621df57-13e9-4afd-898f-1f9adcf61b1e","Type":"ContainerDied","Data":"a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4"}
Jul 02 20:10:32.908741 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.908632    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77558d7458-h8qvs" event={"ID":"2621df57-13e9-4afd-898f-1f9adcf61b1e","Type":"ContainerDied","Data":"5796ee9413ff35eb5c113e37e080366d029ac2dd1245efb21315d7d8d4cb6ad1"}
Jul 02 20:10:32.908741 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.908661    2536 scope.go:117] "RemoveContainer" containerID="a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4"
Jul 02 20:10:32.917174 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.917161    2536 scope.go:117] "RemoveContainer" containerID="a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4"
Jul 02 20:10:32.917422 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:10:32.917407    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4\": container with ID starting with a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4 not found: ID does not exist" containerID="a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4"
Jul 02 20:10:32.917464 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.917429    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4"} err="failed to get container status \"a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4\": rpc error: code = NotFound desc = could not find container \"a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4\": container with ID starting with a06128bf0fe70122a84aeae49891f35277b4cbbde29e9dfb5affc8f9037303e4 not found: ID does not exist"
Jul 02 20:10:32.928510 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.928492    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77558d7458-h8qvs"]
Jul 02 20:10:32.932722 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:32.932704    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77558d7458-h8qvs"]
Jul 02 20:10:33.706254 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.706223    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77558d7458-w89r2"]
Jul 02 20:10:33.706661 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.706601    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2621df57-13e9-4afd-898f-1f9adcf61b1e" containerName="manager"
Jul 02 20:10:33.706661 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.706614    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="2621df57-13e9-4afd-898f-1f9adcf61b1e" containerName="manager"
Jul 02 20:10:33.706739 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.706673    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="2621df57-13e9-4afd-898f-1f9adcf61b1e" containerName="manager"
Jul 02 20:10:33.710835 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.710819    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:33.713996 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.713979    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 02 20:10:33.714080 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.713979    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 02 20:10:33.715043 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.715023    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-crchz\""
Jul 02 20:10:33.720348 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.720329    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77558d7458-w89r2"]
Jul 02 20:10:33.773345 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.773313    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39235ae-9f85-493e-840a-f04b29f4dc35-webhook-cert\") pod \"maas-controller-77558d7458-w89r2\" (UID: \"a39235ae-9f85-493e-840a-f04b29f4dc35\") " pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:33.773467 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.773355    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxx77\" (UniqueName: \"kubernetes.io/projected/a39235ae-9f85-493e-840a-f04b29f4dc35-kube-api-access-bxx77\") pod \"maas-controller-77558d7458-w89r2\" (UID: \"a39235ae-9f85-493e-840a-f04b29f4dc35\") " pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:33.874786 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.874757    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39235ae-9f85-493e-840a-f04b29f4dc35-webhook-cert\") pod \"maas-controller-77558d7458-w89r2\" (UID: \"a39235ae-9f85-493e-840a-f04b29f4dc35\") " pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:33.874917 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.874794    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxx77\" (UniqueName: \"kubernetes.io/projected/a39235ae-9f85-493e-840a-f04b29f4dc35-kube-api-access-bxx77\") pod \"maas-controller-77558d7458-w89r2\" (UID: \"a39235ae-9f85-493e-840a-f04b29f4dc35\") " pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:33.877138 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.877120    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a39235ae-9f85-493e-840a-f04b29f4dc35-webhook-cert\") pod \"maas-controller-77558d7458-w89r2\" (UID: \"a39235ae-9f85-493e-840a-f04b29f4dc35\") " pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:33.883842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:33.883825    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxx77\" (UniqueName: \"kubernetes.io/projected/a39235ae-9f85-493e-840a-f04b29f4dc35-kube-api-access-bxx77\") pod \"maas-controller-77558d7458-w89r2\" (UID: \"a39235ae-9f85-493e-840a-f04b29f4dc35\") " pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:34.021629 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:34.021554    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:34.150326 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:34.150223    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77558d7458-w89r2"]
Jul 02 20:10:34.152450 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:10:34.152427    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39235ae_9f85_493e_840a_f04b29f4dc35.slice/crio-35cab73b6eb047af0f5ce443a850caba06fe1ea4a99b408f17bca5257da4b8b4 WatchSource:0}: Error finding container 35cab73b6eb047af0f5ce443a850caba06fe1ea4a99b408f17bca5257da4b8b4: Status 404 returned error can't find the container with id 35cab73b6eb047af0f5ce443a850caba06fe1ea4a99b408f17bca5257da4b8b4
Jul 02 20:10:34.369163 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:34.369097    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2621df57-13e9-4afd-898f-1f9adcf61b1e" path="/var/lib/kubelet/pods/2621df57-13e9-4afd-898f-1f9adcf61b1e/volumes"
Jul 02 20:10:34.919019 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:34.918987    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77558d7458-w89r2" event={"ID":"a39235ae-9f85-493e-840a-f04b29f4dc35","Type":"ContainerStarted","Data":"a0e1fa5f3a6ca9afa1c28c9984fd10512f37f81a55b766c26f000cadf16a6865"}
Jul 02 20:10:34.919403 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:34.919033    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77558d7458-w89r2" event={"ID":"a39235ae-9f85-493e-840a-f04b29f4dc35","Type":"ContainerStarted","Data":"35cab73b6eb047af0f5ce443a850caba06fe1ea4a99b408f17bca5257da4b8b4"}
Jul 02 20:10:34.919403 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:34.919122    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:10:34.935028 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:34.934974    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77558d7458-w89r2" podStartSLOduration=1.5303571919999999 podStartE2EDuration="1.934961459s" totalImagesPullingTime="404.604267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:10:34.93407563 +0000 UTC m=+847.081844420" watchObservedRunningTime="2026-07-02 20:10:34.934961459 +0000 UTC m=+847.082730236"
Jul 02 20:10:45.930615 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:10:45.930580    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77558d7458-w89r2"
Jul 02 20:22:33.191069 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.191028    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"]
Jul 02 20:22:33.194539 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.194519    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.198350 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.198329    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-18260bb7-openshift-default-dockercfg-bvdk7\""
Jul 02 20:22:33.296514 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.296473    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"]
Jul 02 20:22:33.357649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357610    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-495pf\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-kube-api-access-495pf\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357659    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67379639-95aa-487a-be11-a8764fa0c195-istio-podinfo\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357710    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-istio-token\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357799    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-credential-socket\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357832    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-data\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357857    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67379639-95aa-487a-be11-a8764fa0c195-istiod-ca-cert\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357876    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-socket\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357907    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-certs\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.357934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.357921    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-envoy\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.458771 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.458664    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-495pf\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-kube-api-access-495pf\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.458771 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.458713    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67379639-95aa-487a-be11-a8764fa0c195-istio-podinfo\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.458837    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-istio-token\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.458927    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-credential-socket\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459137 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459085    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-data\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459193 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459138    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67379639-95aa-487a-be11-a8764fa0c195-istiod-ca-cert\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459193 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459170    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-socket\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459343 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459239    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-credential-socket\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459343 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459311    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-certs\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459460 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459344    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-envoy\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459471    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-data\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459506    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-socket\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459695 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459670    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-certs\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.459870 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.459848    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67379639-95aa-487a-be11-a8764fa0c195-istiod-ca-cert\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.461482 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.461461    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67379639-95aa-487a-be11-a8764fa0c195-istio-podinfo\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.461735 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.461715    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-envoy\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.467494 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.467469    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-istio-token\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.467818 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.467799    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-495pf\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-kube-api-access-495pf\") pod \"e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") " pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.505762 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.505724    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:33.656681 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.656655    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"]
Jul 02 20:22:33.658635 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:22:33.658603    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67379639_95aa_487a_be11_a8764fa0c195.slice/crio-cc33ef6e1bc2f5bd24f8285ef144f0f34da02a116a6aefa94edceda9fdf66fdc WatchSource:0}: Error finding container cc33ef6e1bc2f5bd24f8285ef144f0f34da02a116a6aefa94edceda9fdf66fdc: Status 404 returned error can't find the container with id cc33ef6e1bc2f5bd24f8285ef144f0f34da02a116a6aefa94edceda9fdf66fdc
Jul 02 20:22:33.660961 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.660925    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:22:33.661075 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.661004    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:22:33.661075 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:33.661038    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:22:34.278732 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.278689    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6" event={"ID":"67379639-95aa-487a-be11-a8764fa0c195","Type":"ContainerStarted","Data":"c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3"}
Jul 02 20:22:34.279150 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.278751    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6" event={"ID":"67379639-95aa-487a-be11-a8764fa0c195","Type":"ContainerStarted","Data":"cc33ef6e1bc2f5bd24f8285ef144f0f34da02a116a6aefa94edceda9fdf66fdc"}
Jul 02 20:22:34.300928 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.300874    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6" podStartSLOduration=1.300856236 podStartE2EDuration="1.300856236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:22:34.298914305 +0000 UTC m=+1566.446683097" watchObservedRunningTime="2026-07-02 20:22:34.300856236 +0000 UTC m=+1566.448625015"
Jul 02 20:22:34.506423 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.506379    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:34.511920 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.511889    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:34.584493 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.584403    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"]
Jul 02 20:22:34.588134 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.588108    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.590593 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.590564    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-18260bb7-serving-cert\""
Jul 02 20:22:34.599155 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.599126    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"]
Jul 02 20:22:34.772072 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.772042    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c160fd-4329-49d4-a259-7bbd84ea801e-openshift-service-ca\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.772248 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.772096    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.772248 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.772200    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sltp\" (UniqueName: \"kubernetes.io/projected/b0c160fd-4329-49d4-a259-7bbd84ea801e-kube-api-access-9sltp\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.873825 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.873668    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c160fd-4329-49d4-a259-7bbd84ea801e-openshift-service-ca\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.874136 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.874104    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.874453 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.874427    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sltp\" (UniqueName: \"kubernetes.io/projected/b0c160fd-4329-49d4-a259-7bbd84ea801e-kube-api-access-9sltp\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.874579 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.874554    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c160fd-4329-49d4-a259-7bbd84ea801e-openshift-service-ca\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:34.877317 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:22:34.874935    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-18260bb7-serving-cert: secret "maas-api-e2e-ait-18260bb7-serving-cert" not found
Jul 02 20:22:34.877317 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:22:34.875020    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls podName:b0c160fd-4329-49d4-a259-7bbd84ea801e nodeName:}" failed. No retries permitted until 2026-07-02 20:22:35.37499997 +0000 UTC m=+1567.522768747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls") pod "maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n" (UID: "b0c160fd-4329-49d4-a259-7bbd84ea801e") : secret "maas-api-e2e-ait-18260bb7-serving-cert" not found
Jul 02 20:22:34.883865 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:34.883836    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sltp\" (UniqueName: \"kubernetes.io/projected/b0c160fd-4329-49d4-a259-7bbd84ea801e-kube-api-access-9sltp\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:35.284143 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:35.284114    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:35.285345 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:35.285317    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:35.380271 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:35.380231    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:35.383138 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:35.383111    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls\") pod \"maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") " pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:35.501180 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:35.501145    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:35.645636 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:35.645540    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"]
Jul 02 20:22:35.648979 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:22:35.648933    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c160fd_4329_49d4_a259_7bbd84ea801e.slice/crio-85cc01e397760fb13c7a5777aba5c04097695816c28f052363b12eb358d96775 WatchSource:0}: Error finding container 85cc01e397760fb13c7a5777aba5c04097695816c28f052363b12eb358d96775: Status 404 returned error can't find the container with id 85cc01e397760fb13c7a5777aba5c04097695816c28f052363b12eb358d96775
Jul 02 20:22:35.650934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:35.650907    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 20:22:36.289852 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:36.289803    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n" event={"ID":"b0c160fd-4329-49d4-a259-7bbd84ea801e","Type":"ContainerStarted","Data":"36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f"}
Jul 02 20:22:36.290417 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:36.289868    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n" event={"ID":"b0c160fd-4329-49d4-a259-7bbd84ea801e","Type":"ContainerStarted","Data":"85cc01e397760fb13c7a5777aba5c04097695816c28f052363b12eb358d96775"}
Jul 02 20:22:36.290417 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:36.290119    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:36.309177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:36.309126    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n" podStartSLOduration=1.773900413 podStartE2EDuration="2.309109647s" totalImagesPullingTime="535.209234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:22:36.306829635 +0000 UTC m=+1568.454598427" watchObservedRunningTime="2026-07-02 20:22:36.309109647 +0000 UTC m=+1568.456878438"
Jul 02 20:22:40.045483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.045451    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"]
Jul 02 20:22:40.046009 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.045628    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6" podUID="67379639-95aa-487a-be11-a8764fa0c195" containerName="istio-proxy" containerID="cri-o://c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3" gracePeriod=30
Jul 02 20:22:40.297076 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.296989    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"]
Jul 02 20:22:40.297248 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.297219    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n" podUID="b0c160fd-4329-49d4-a259-7bbd84ea801e" containerName="maas-api" containerID="cri-o://36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f" gracePeriod=30
Jul 02 20:22:40.598412 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.598384    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:40.625976 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.625949    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sltp\" (UniqueName: \"kubernetes.io/projected/b0c160fd-4329-49d4-a259-7bbd84ea801e-kube-api-access-9sltp\") pod \"b0c160fd-4329-49d4-a259-7bbd84ea801e\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") "
Jul 02 20:22:40.626176 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.626041    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c160fd-4329-49d4-a259-7bbd84ea801e-openshift-service-ca\") pod \"b0c160fd-4329-49d4-a259-7bbd84ea801e\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") "
Jul 02 20:22:40.628749 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.627054    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0c160fd-4329-49d4-a259-7bbd84ea801e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0c160fd-4329-49d4-a259-7bbd84ea801e" (UID: "b0c160fd-4329-49d4-a259-7bbd84ea801e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:22:40.633337 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.631409    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0c160fd-4329-49d4-a259-7bbd84ea801e-kube-api-access-9sltp" (OuterVolumeSpecName: "kube-api-access-9sltp") pod "b0c160fd-4329-49d4-a259-7bbd84ea801e" (UID: "b0c160fd-4329-49d4-a259-7bbd84ea801e"). InnerVolumeSpecName "kube-api-access-9sltp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:22:40.727113 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.727077    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls\") pod \"b0c160fd-4329-49d4-a259-7bbd84ea801e\" (UID: \"b0c160fd-4329-49d4-a259-7bbd84ea801e\") "
Jul 02 20:22:40.727278 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.727237    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sltp\" (UniqueName: \"kubernetes.io/projected/b0c160fd-4329-49d4-a259-7bbd84ea801e-kube-api-access-9sltp\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:40.727278 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.727249    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c160fd-4329-49d4-a259-7bbd84ea801e-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:40.729604 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.729566    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0c160fd-4329-49d4-a259-7bbd84ea801e" (UID: "b0c160fd-4329-49d4-a259-7bbd84ea801e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:22:40.827859 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:40.827749    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0c160fd-4329-49d4-a259-7bbd84ea801e-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:41.315750 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.315716    2536 generic.go:356] "Generic (PLEG): container finished" podID="b0c160fd-4329-49d4-a259-7bbd84ea801e" containerID="36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f" exitCode=0
Jul 02 20:22:41.316161 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.315764    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"
Jul 02 20:22:41.316161 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.315803    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n" event={"ID":"b0c160fd-4329-49d4-a259-7bbd84ea801e","Type":"ContainerDied","Data":"36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f"}
Jul 02 20:22:41.316161 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.315887    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n" event={"ID":"b0c160fd-4329-49d4-a259-7bbd84ea801e","Type":"ContainerDied","Data":"85cc01e397760fb13c7a5777aba5c04097695816c28f052363b12eb358d96775"}
Jul 02 20:22:41.316161 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.315911    2536 scope.go:117] "RemoveContainer" containerID="36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f"
Jul 02 20:22:41.324788 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.324771    2536 scope.go:117] "RemoveContainer" containerID="36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f"
Jul 02 20:22:41.325055 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:22:41.325036    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f\": container with ID starting with 36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f not found: ID does not exist" containerID="36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f"
Jul 02 20:22:41.325102 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.325064    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f"} err="failed to get container status \"36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f\": rpc error: code = NotFound desc = could not find container \"36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f\": container with ID starting with 36e4afd72516044c1a6f38529c3f7fc96b9da319907c99279fac72549b62090f not found: ID does not exist"
Jul 02 20:22:41.350316 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.349785    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"]
Jul 02 20:22:41.366935 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:41.366899    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-18260bb7-6f4cf4bc98-qkq6n"]
Jul 02 20:22:42.370237 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:42.370204    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0c160fd-4329-49d4-a259-7bbd84ea801e" path="/var/lib/kubelet/pods/b0c160fd-4329-49d4-a259-7bbd84ea801e/volumes"
Jul 02 20:22:44.719238 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.719215    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:44.756855 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.756821    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-socket\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.756903    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-credential-socket\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.756924    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-istio-token\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.756940    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-data\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.756960    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-certs\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.756978    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67379639-95aa-487a-be11-a8764fa0c195-istiod-ca-cert\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757021    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-envoy\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757354 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757118    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-495pf\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-kube-api-access-495pf\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757354 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757122    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:44.757354 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757155    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67379639-95aa-487a-be11-a8764fa0c195-istio-podinfo\") pod \"67379639-95aa-487a-be11-a8764fa0c195\" (UID: \"67379639-95aa-487a-be11-a8764fa0c195\") "
Jul 02 20:22:44.757354 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757230    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:44.757354 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757269    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:44.757585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757486    2536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.757585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757505    2536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-credential-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.757585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757522    2536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-workload-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.757739 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757674    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67379639-95aa-487a-be11-a8764fa0c195-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:22:44.757935 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.757906    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-data" (OuterVolumeSpecName: "istio-data") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:44.759817 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.759794    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67379639-95aa-487a-be11-a8764fa0c195-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 20:22:44.759997 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.759978    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-istio-token" (OuterVolumeSpecName: "istio-token") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:22:44.760145 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.760125    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-kube-api-access-495pf" (OuterVolumeSpecName: "kube-api-access-495pf") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "kube-api-access-495pf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:22:44.760202 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.760163    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67379639-95aa-487a-be11-a8764fa0c195" (UID: "67379639-95aa-487a-be11-a8764fa0c195"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:44.857994 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.857890    2536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-istio-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.857994 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.857922    2536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-data\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.857994 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.857935    2536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67379639-95aa-487a-be11-a8764fa0c195-istiod-ca-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.857994 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.857946    2536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67379639-95aa-487a-be11-a8764fa0c195-istio-envoy\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.857994 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.857956    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-495pf\" (UniqueName: \"kubernetes.io/projected/67379639-95aa-487a-be11-a8764fa0c195-kube-api-access-495pf\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:44.857994 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:44.857969    2536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67379639-95aa-487a-be11-a8764fa0c195-istio-podinfo\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:45.339426 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.339390    2536 generic.go:356] "Generic (PLEG): container finished" podID="67379639-95aa-487a-be11-a8764fa0c195" containerID="c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3" exitCode=0
Jul 02 20:22:45.339625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.339440    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"
Jul 02 20:22:45.339625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.339470    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6" event={"ID":"67379639-95aa-487a-be11-a8764fa0c195","Type":"ContainerDied","Data":"c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3"}
Jul 02 20:22:45.339625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.339524    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6" event={"ID":"67379639-95aa-487a-be11-a8764fa0c195","Type":"ContainerDied","Data":"cc33ef6e1bc2f5bd24f8285ef144f0f34da02a116a6aefa94edceda9fdf66fdc"}
Jul 02 20:22:45.339625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.339548    2536 scope.go:117] "RemoveContainer" containerID="c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3"
Jul 02 20:22:45.349522 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.349506    2536 scope.go:117] "RemoveContainer" containerID="c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3"
Jul 02 20:22:45.349757 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:22:45.349742    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3\": container with ID starting with c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3 not found: ID does not exist" containerID="c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3"
Jul 02 20:22:45.349808 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.349768    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3"} err="failed to get container status \"c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3\": rpc error: code = NotFound desc = could not find container \"c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3\": container with ID starting with c57c5268f55544e72061efcda4f0751192424cdb6f84da314d6befb2bc12ccf3 not found: ID does not exist"
Jul 02 20:22:45.361872 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.361844    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"]
Jul 02 20:22:45.365851 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:45.365826    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-18260bb7-openshift-default-6775c954c5-qzls6"]
Jul 02 20:22:46.204793 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.204762    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"]
Jul 02 20:22:46.205171 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.205145    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67379639-95aa-487a-be11-a8764fa0c195" containerName="istio-proxy"
Jul 02 20:22:46.205171 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.205156    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="67379639-95aa-487a-be11-a8764fa0c195" containerName="istio-proxy"
Jul 02 20:22:46.205243 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.205173    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0c160fd-4329-49d4-a259-7bbd84ea801e" containerName="maas-api"
Jul 02 20:22:46.205243 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.205179    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0c160fd-4329-49d4-a259-7bbd84ea801e" containerName="maas-api"
Jul 02 20:22:46.205404 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.205255    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="67379639-95aa-487a-be11-a8764fa0c195" containerName="istio-proxy"
Jul 02 20:22:46.205404 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.205265    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0c160fd-4329-49d4-a259-7bbd84ea801e" containerName="maas-api"
Jul 02 20:22:46.209732 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.209707    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.212500 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.212477    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d53ccb06-openshift-default-dockercfg-29r7d\""
Jul 02 20:22:46.231618 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.231589    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"]
Jul 02 20:22:46.270086 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270054    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-certs\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270099    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-socket\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270124    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-credential-socket\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270152    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-envoy\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270184    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-token\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270261    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-podinfo\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270491 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270307    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-data\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270491 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270330    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94055166-bb8c-4512-a8bb-d684cca4d7ce-istiod-ca-cert\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.270491 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.270354    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fxp4\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-kube-api-access-9fxp4\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371159 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371124    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-podinfo\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371159 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371165    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-data\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371489 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371188    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94055166-bb8c-4512-a8bb-d684cca4d7ce-istiod-ca-cert\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371489 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371373    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fxp4\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-kube-api-access-9fxp4\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371489 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371483    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-certs\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371648 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371527    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-socket\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371648 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371559    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-credential-socket\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371648 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371589    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-envoy\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.371648 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371635    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-token\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.372181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.372043    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94055166-bb8c-4512-a8bb-d684cca4d7ce-istiod-ca-cert\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.372181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.372090    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-credential-socket\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.372495 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.371634    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-data\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.372495 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.372311    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-certs\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.372495 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.372333    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-socket\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.380768 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.374622    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-podinfo\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.380768 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.375080    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67379639-95aa-487a-be11-a8764fa0c195" path="/var/lib/kubelet/pods/67379639-95aa-487a-be11-a8764fa0c195/volumes"
Jul 02 20:22:46.381413 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.381378    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-envoy\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.396238 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.396208    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fxp4\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-kube-api-access-9fxp4\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.397618 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.397589    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-token\") pod \"e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") " pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.522965 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.522866    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:46.685537 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.685510    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"]
Jul 02 20:22:46.687822 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:22:46.687795    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94055166_bb8c_4512_a8bb_d684cca4d7ce.slice/crio-9bd60ed01965421e844ba53b6c2f5c70eb80a4d37c70d0ba78e11cd8b634c45c WatchSource:0}: Error finding container 9bd60ed01965421e844ba53b6c2f5c70eb80a4d37c70d0ba78e11cd8b634c45c: Status 404 returned error can't find the container with id 9bd60ed01965421e844ba53b6c2f5c70eb80a4d37c70d0ba78e11cd8b634c45c
Jul 02 20:22:46.689842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.689808    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:22:46.689932 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.689878    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:22:46.689932 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:46.689906    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:22:47.352021 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.351976    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t" event={"ID":"94055166-bb8c-4512-a8bb-d684cca4d7ce","Type":"ContainerStarted","Data":"36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210"}
Jul 02 20:22:47.352447 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.352054    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t" event={"ID":"94055166-bb8c-4512-a8bb-d684cca4d7ce","Type":"ContainerStarted","Data":"9bd60ed01965421e844ba53b6c2f5c70eb80a4d37c70d0ba78e11cd8b634c45c"}
Jul 02 20:22:47.373626 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.373580    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t" podStartSLOduration=1.373561632 podStartE2EDuration="1.373561632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:22:47.371332474 +0000 UTC m=+1579.519101252" watchObservedRunningTime="2026-07-02 20:22:47.373561632 +0000 UTC m=+1579.521330411"
Jul 02 20:22:47.523545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.523503    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:47.528657 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.528629    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:47.681998 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.681966    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"]
Jul 02 20:22:47.685922 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.685898    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.688904 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.688882    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-d53ccb06-serving-cert\""
Jul 02 20:22:47.699402 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.699350    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"]
Jul 02 20:22:47.781036 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.781002    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/237dc002-ae6c-4306-9d9c-95ad3e4be83a-openshift-service-ca\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.781211 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.781048    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5d8d\" (UniqueName: \"kubernetes.io/projected/237dc002-ae6c-4306-9d9c-95ad3e4be83a-kube-api-access-k5d8d\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.781211 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.781102    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.881601 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.881563    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/237dc002-ae6c-4306-9d9c-95ad3e4be83a-openshift-service-ca\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.881601 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.881606    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5d8d\" (UniqueName: \"kubernetes.io/projected/237dc002-ae6c-4306-9d9c-95ad3e4be83a-kube-api-access-k5d8d\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.881819 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.881639    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.881819 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:22:47.881778    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-d53ccb06-serving-cert: secret "maas-api-e2e-ait-d53ccb06-serving-cert" not found
Jul 02 20:22:47.881888 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:22:47.881839    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls podName:237dc002-ae6c-4306-9d9c-95ad3e4be83a nodeName:}" failed. No retries permitted until 2026-07-02 20:22:48.381819603 +0000 UTC m=+1580.529588361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls") pod "maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr" (UID: "237dc002-ae6c-4306-9d9c-95ad3e4be83a") : secret "maas-api-e2e-ait-d53ccb06-serving-cert" not found
Jul 02 20:22:47.882195 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.882176    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/237dc002-ae6c-4306-9d9c-95ad3e4be83a-openshift-service-ca\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:47.894460 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:47.894436    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5d8d\" (UniqueName: \"kubernetes.io/projected/237dc002-ae6c-4306-9d9c-95ad3e4be83a-kube-api-access-k5d8d\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:48.357623 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:48.357589    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:48.358763 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:48.358740    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:48.385847 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:48.385814    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:48.388472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:48.388447    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls\") pod \"maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") " pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:48.597404 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:48.597066    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:48.753747 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:48.753714    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"]
Jul 02 20:22:48.757158 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:22:48.757133    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237dc002_ae6c_4306_9d9c_95ad3e4be83a.slice/crio-adccd74ca590fb3b5de76783646a34d52321672ef7cd50fde84175f9c22c22e3 WatchSource:0}: Error finding container adccd74ca590fb3b5de76783646a34d52321672ef7cd50fde84175f9c22c22e3: Status 404 returned error can't find the container with id adccd74ca590fb3b5de76783646a34d52321672ef7cd50fde84175f9c22c22e3
Jul 02 20:22:49.366103 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:49.366017    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr" event={"ID":"237dc002-ae6c-4306-9d9c-95ad3e4be83a","Type":"ContainerStarted","Data":"cfc14e40bd4a83dabaf5cdf94585801d65b1b525df7f0c704d48079c2f95d601"}
Jul 02 20:22:49.366103 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:49.366070    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr" event={"ID":"237dc002-ae6c-4306-9d9c-95ad3e4be83a","Type":"ContainerStarted","Data":"adccd74ca590fb3b5de76783646a34d52321672ef7cd50fde84175f9c22c22e3"}
Jul 02 20:22:49.391302 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:49.391236    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr" podStartSLOduration=2.043789132 podStartE2EDuration="2.391218461s" totalImagesPullingTime="347.429329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:22:49.389568919 +0000 UTC m=+1581.537337699" watchObservedRunningTime="2026-07-02 20:22:49.391218461 +0000 UTC m=+1581.538987239"
Jul 02 20:22:50.371609 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:50.371578    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:53.186090 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.186052    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"]
Jul 02 20:22:53.186600 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.186324    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr" podUID="237dc002-ae6c-4306-9d9c-95ad3e4be83a" containerName="maas-api" containerID="cri-o://cfc14e40bd4a83dabaf5cdf94585801d65b1b525df7f0c704d48079c2f95d601" gracePeriod=30
Jul 02 20:22:53.394672 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.394624    2536 generic.go:356] "Generic (PLEG): container finished" podID="237dc002-ae6c-4306-9d9c-95ad3e4be83a" containerID="cfc14e40bd4a83dabaf5cdf94585801d65b1b525df7f0c704d48079c2f95d601" exitCode=0
Jul 02 20:22:53.394847 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.394697    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr" event={"ID":"237dc002-ae6c-4306-9d9c-95ad3e4be83a","Type":"ContainerDied","Data":"cfc14e40bd4a83dabaf5cdf94585801d65b1b525df7f0c704d48079c2f95d601"}
Jul 02 20:22:53.479993 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.479960    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:53.528300 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.528256    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5d8d\" (UniqueName: \"kubernetes.io/projected/237dc002-ae6c-4306-9d9c-95ad3e4be83a-kube-api-access-k5d8d\") pod \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") "
Jul 02 20:22:53.528477 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.528414    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/237dc002-ae6c-4306-9d9c-95ad3e4be83a-openshift-service-ca\") pod \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") "
Jul 02 20:22:53.528477 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.528469    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls\") pod \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\" (UID: \"237dc002-ae6c-4306-9d9c-95ad3e4be83a\") "
Jul 02 20:22:53.528803 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.528780    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/237dc002-ae6c-4306-9d9c-95ad3e4be83a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "237dc002-ae6c-4306-9d9c-95ad3e4be83a" (UID: "237dc002-ae6c-4306-9d9c-95ad3e4be83a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:22:53.530725 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.530701    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/237dc002-ae6c-4306-9d9c-95ad3e4be83a-kube-api-access-k5d8d" (OuterVolumeSpecName: "kube-api-access-k5d8d") pod "237dc002-ae6c-4306-9d9c-95ad3e4be83a" (UID: "237dc002-ae6c-4306-9d9c-95ad3e4be83a"). InnerVolumeSpecName "kube-api-access-k5d8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:22:53.530725 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.530717    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "237dc002-ae6c-4306-9d9c-95ad3e4be83a" (UID: "237dc002-ae6c-4306-9d9c-95ad3e4be83a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:22:53.629579 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.629528    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/237dc002-ae6c-4306-9d9c-95ad3e4be83a-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:53.629579 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.629573    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/237dc002-ae6c-4306-9d9c-95ad3e4be83a-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:53.629579 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:53.629583    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5d8d\" (UniqueName: \"kubernetes.io/projected/237dc002-ae6c-4306-9d9c-95ad3e4be83a-kube-api-access-k5d8d\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:54.401227 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:54.401192    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr" event={"ID":"237dc002-ae6c-4306-9d9c-95ad3e4be83a","Type":"ContainerDied","Data":"adccd74ca590fb3b5de76783646a34d52321672ef7cd50fde84175f9c22c22e3"}
Jul 02 20:22:54.401227 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:54.401210    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"
Jul 02 20:22:54.401750 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:54.401253    2536 scope.go:117] "RemoveContainer" containerID="cfc14e40bd4a83dabaf5cdf94585801d65b1b525df7f0c704d48079c2f95d601"
Jul 02 20:22:54.419996 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:54.419968    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"]
Jul 02 20:22:54.423605 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:54.423582    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-d53ccb06-77dd7bb9b-r94rr"]
Jul 02 20:22:54.696693 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:54.696661    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"]
Jul 02 20:22:54.696901 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:54.696853    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t" podUID="94055166-bb8c-4512-a8bb-d684cca4d7ce" containerName="istio-proxy" containerID="cri-o://36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210" gracePeriod=30
Jul 02 20:22:56.373890 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:56.373852    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="237dc002-ae6c-4306-9d9c-95ad3e4be83a" path="/var/lib/kubelet/pods/237dc002-ae6c-4306-9d9c-95ad3e4be83a/volumes"
Jul 02 20:22:58.765045 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.765016    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:58.878713 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878623    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-token\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.878874 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878738    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-envoy\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.878874 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878772    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-podinfo\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.878874 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878801    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-credential-socket\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.878874 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878818    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fxp4\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-kube-api-access-9fxp4\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.878874 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878842    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-data\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.879128 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878888    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94055166-bb8c-4512-a8bb-d684cca4d7ce-istiod-ca-cert\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.879128 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878927    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-certs\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.879128 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.878953    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-socket\") pod \"94055166-bb8c-4512-a8bb-d684cca4d7ce\" (UID: \"94055166-bb8c-4512-a8bb-d684cca4d7ce\") "
Jul 02 20:22:58.879332 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.879125    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:58.879332 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.879324    2536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-credential-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.879444 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.879383    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:58.879551 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.879515    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:58.879738 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.879434    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94055166-bb8c-4512-a8bb-d684cca4d7ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:22:58.880142 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.880098    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:58.881092 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.881062    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:22:58.881234 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.881213    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 20:22:58.881517 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.881243    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-kube-api-access-9fxp4" (OuterVolumeSpecName: "kube-api-access-9fxp4") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "kube-api-access-9fxp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:22:58.881644 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.881578    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "94055166-bb8c-4512-a8bb-d684cca4d7ce" (UID: "94055166-bb8c-4512-a8bb-d684cca4d7ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:22:58.980597 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980557    2536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94055166-bb8c-4512-a8bb-d684cca4d7ce-istiod-ca-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.980597 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980590    2536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.980597 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980600    2536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-workload-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.980844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980609    2536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.980844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980619    2536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-envoy\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.980844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980627    2536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-podinfo\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.980844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980635    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fxp4\" (UniqueName: \"kubernetes.io/projected/94055166-bb8c-4512-a8bb-d684cca4d7ce-kube-api-access-9fxp4\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:58.980844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:58.980643    2536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94055166-bb8c-4512-a8bb-d684cca4d7ce-istio-data\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:22:59.429526 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.429491    2536 generic.go:356] "Generic (PLEG): container finished" podID="94055166-bb8c-4512-a8bb-d684cca4d7ce" containerID="36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210" exitCode=0
Jul 02 20:22:59.429739 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.429543    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"
Jul 02 20:22:59.429739 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.429573    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t" event={"ID":"94055166-bb8c-4512-a8bb-d684cca4d7ce","Type":"ContainerDied","Data":"36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210"}
Jul 02 20:22:59.429739 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.429621    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t" event={"ID":"94055166-bb8c-4512-a8bb-d684cca4d7ce","Type":"ContainerDied","Data":"9bd60ed01965421e844ba53b6c2f5c70eb80a4d37c70d0ba78e11cd8b634c45c"}
Jul 02 20:22:59.429739 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.429643    2536 scope.go:117] "RemoveContainer" containerID="36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210"
Jul 02 20:22:59.439261 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.439244    2536 scope.go:117] "RemoveContainer" containerID="36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210"
Jul 02 20:22:59.439570 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:22:59.439552    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210\": container with ID starting with 36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210 not found: ID does not exist" containerID="36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210"
Jul 02 20:22:59.439613 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.439578    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210"} err="failed to get container status \"36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210\": rpc error: code = NotFound desc = could not find container \"36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210\": container with ID starting with 36b8948f0c6dfb9ceb9e10d122cd7a25951bcdfca05950d981eb0e9fc4a97210 not found: ID does not exist"
Jul 02 20:22:59.451038 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.451016    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"]
Jul 02 20:22:59.457219 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:22:59.457190    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d53ccb06-openshift-default-864bd7fd67-gjn7t"]
Jul 02 20:23:00.369175 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.369139    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94055166-bb8c-4512-a8bb-d684cca4d7ce" path="/var/lib/kubelet/pods/94055166-bb8c-4512-a8bb-d684cca4d7ce/volumes"
Jul 02 20:23:00.993885 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.993842    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"]
Jul 02 20:23:00.994472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.994445    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="237dc002-ae6c-4306-9d9c-95ad3e4be83a" containerName="maas-api"
Jul 02 20:23:00.994472 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.994473    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="237dc002-ae6c-4306-9d9c-95ad3e4be83a" containerName="maas-api"
Jul 02 20:23:00.994677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.994509    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94055166-bb8c-4512-a8bb-d684cca4d7ce" containerName="istio-proxy"
Jul 02 20:23:00.994677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.994518    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="94055166-bb8c-4512-a8bb-d684cca4d7ce" containerName="istio-proxy"
Jul 02 20:23:00.994677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.994615    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="237dc002-ae6c-4306-9d9c-95ad3e4be83a" containerName="maas-api"
Jul 02 20:23:00.994677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.994627    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="94055166-bb8c-4512-a8bb-d684cca4d7ce" containerName="istio-proxy"
Jul 02 20:23:00.999511 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:00.999487    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.002052 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.002030    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ffaf0090-openshift-default-dockercfg-6rfhr\""
Jul 02 20:23:01.012186 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.012155    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"]
Jul 02 20:23:01.102263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102231    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istiod-ca-cert\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102267    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-certs\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102566 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102352    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-data\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102566 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102384    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-token\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102566 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102486    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-socket\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102566 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102518    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq58v\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-kube-api-access-hq58v\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102733 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102581    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-credential-socket\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102733 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102650    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-envoy\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.102733 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.102674    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-podinfo\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.203976 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.203930    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-envoy\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.203976 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.203978    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-podinfo\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204275 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204030    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istiod-ca-cert\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204275 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204057    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-certs\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204275 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204087    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-data\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204275 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204113    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-token\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204275 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204262    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-socket\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204675 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204332    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq58v\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-kube-api-access-hq58v\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204675 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204393    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-credential-socket\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204675 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204514    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-data\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204675 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204524    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-certs\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.204675 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204600    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-socket\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.205091 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204724    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-credential-socket\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.205091 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.204992    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istiod-ca-cert\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.206553 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.206528    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-envoy\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.206818 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.206802    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-podinfo\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.227198 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.227165    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-token\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.227398 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.227381    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq58v\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-kube-api-access-hq58v\") pod \"e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") " pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.311857 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.311760    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:01.458751 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.458727    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"]
Jul 02 20:23:01.460278 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:23:01.460246    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b3e1c44_e7cd_41ce_8667_952e5a2ca1cd.slice/crio-80b84930300a0a0a73318494136f706385a19142534e5869ff6889b2ca2ec1c4 WatchSource:0}: Error finding container 80b84930300a0a0a73318494136f706385a19142534e5869ff6889b2ca2ec1c4: Status 404 returned error can't find the container with id 80b84930300a0a0a73318494136f706385a19142534e5869ff6889b2ca2ec1c4
Jul 02 20:23:01.462427 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.462398    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:23:01.462494 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.462469    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:23:01.462532 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:01.462501    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:23:02.313406 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.313369    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"]
Jul 02 20:23:02.317208 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.317186    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.319944 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.319923    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-ffaf0090-serving-cert\""
Jul 02 20:23:02.327025 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.326993    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"]
Jul 02 20:23:02.416365 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.416329    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-openshift-service-ca\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.416539 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.416423    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6zx5\" (UniqueName: \"kubernetes.io/projected/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-kube-api-access-m6zx5\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.416539 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.416466    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.446434 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.446398    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f" event={"ID":"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd","Type":"ContainerStarted","Data":"c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599"}
Jul 02 20:23:02.446613 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.446459    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f" event={"ID":"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd","Type":"ContainerStarted","Data":"80b84930300a0a0a73318494136f706385a19142534e5869ff6889b2ca2ec1c4"}
Jul 02 20:23:02.465187 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.465130    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f" podStartSLOduration=2.465113389 podStartE2EDuration="2.465113389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:23:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:23:02.46465986 +0000 UTC m=+1594.612428643" watchObservedRunningTime="2026-07-02 20:23:02.465113389 +0000 UTC m=+1594.612882164"
Jul 02 20:23:02.517843 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.517802    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6zx5\" (UniqueName: \"kubernetes.io/projected/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-kube-api-access-m6zx5\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.518026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.517850    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.518026 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:23:02.517944    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-ffaf0090-serving-cert: secret "maas-api-e2e-derive-ffaf0090-serving-cert" not found
Jul 02 20:23:02.518026 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.517986    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-openshift-service-ca\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.518026 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:23:02.518014    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls podName:ff9d33cf-b02a-48b2-a8d7-4f3b38330369 nodeName:}" failed. No retries permitted until 2026-07-02 20:23:03.017993968 +0000 UTC m=+1595.165762727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls") pod "maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k" (UID: "ff9d33cf-b02a-48b2-a8d7-4f3b38330369") : secret "maas-api-e2e-derive-ffaf0090-serving-cert" not found
Jul 02 20:23:02.518696 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.518669    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-openshift-service-ca\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:02.531291 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:02.531262    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6zx5\" (UniqueName: \"kubernetes.io/projected/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-kube-api-access-m6zx5\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:03.022424 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.022382    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:03.025479 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.025404    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls\") pod \"maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") " pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:03.232302 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.232257    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:03.311919 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.311889    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:03.317775 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.317751    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:03.381699 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.381677    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"]
Jul 02 20:23:03.383124 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:23:03.383077    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9d33cf_b02a_48b2_a8d7_4f3b38330369.slice/crio-a0da59ed6641ff45702e7bea5c79ed2f3e9d440a66ba547fd91e8a54c4ccec0e WatchSource:0}: Error finding container a0da59ed6641ff45702e7bea5c79ed2f3e9d440a66ba547fd91e8a54c4ccec0e: Status 404 returned error can't find the container with id a0da59ed6641ff45702e7bea5c79ed2f3e9d440a66ba547fd91e8a54c4ccec0e
Jul 02 20:23:03.453515 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.453479    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k" event={"ID":"ff9d33cf-b02a-48b2-a8d7-4f3b38330369","Type":"ContainerStarted","Data":"a0da59ed6641ff45702e7bea5c79ed2f3e9d440a66ba547fd91e8a54c4ccec0e"}
Jul 02 20:23:03.453769 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.453749    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:03.454749 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:03.454728    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:04.461448 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:04.461413    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k" event={"ID":"ff9d33cf-b02a-48b2-a8d7-4f3b38330369","Type":"ContainerStarted","Data":"b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7"}
Jul 02 20:23:04.480597 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:04.480547    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k" podStartSLOduration=2.117835114 podStartE2EDuration="2.480530261s" totalImagesPullingTime="362.695147ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:23:04.479062437 +0000 UTC m=+1596.626831215" watchObservedRunningTime="2026-07-02 20:23:04.480530261 +0000 UTC m=+1596.628299040"
Jul 02 20:23:05.467083 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:05.467047    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:06.876585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:06.876547    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"]
Jul 02 20:23:06.877074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:06.876794    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f" podUID="8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" containerName="istio-proxy" containerID="cri-o://c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599" gracePeriod=30
Jul 02 20:23:07.085271 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.085228    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"]
Jul 02 20:23:07.479630 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.479590    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k" podUID="ff9d33cf-b02a-48b2-a8d7-4f3b38330369" containerName="maas-api" containerID="cri-o://b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7" gracePeriod=30
Jul 02 20:23:07.779090 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.779056    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:07.891186 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.891153    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-openshift-service-ca\") pod \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") "
Jul 02 20:23:07.891693 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.891204    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls\") pod \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") "
Jul 02 20:23:07.891693 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.891249    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6zx5\" (UniqueName: \"kubernetes.io/projected/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-kube-api-access-m6zx5\") pod \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\" (UID: \"ff9d33cf-b02a-48b2-a8d7-4f3b38330369\") "
Jul 02 20:23:07.891693 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.891502    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff9d33cf-b02a-48b2-a8d7-4f3b38330369" (UID: "ff9d33cf-b02a-48b2-a8d7-4f3b38330369"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:23:07.893475 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.893453    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff9d33cf-b02a-48b2-a8d7-4f3b38330369" (UID: "ff9d33cf-b02a-48b2-a8d7-4f3b38330369"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:23:07.893613 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.893593    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-kube-api-access-m6zx5" (OuterVolumeSpecName: "kube-api-access-m6zx5") pod "ff9d33cf-b02a-48b2-a8d7-4f3b38330369" (UID: "ff9d33cf-b02a-48b2-a8d7-4f3b38330369"). InnerVolumeSpecName "kube-api-access-m6zx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:23:07.992229 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.992136    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:07.992229 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.992167    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:07.992229 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:07.992176    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6zx5\" (UniqueName: \"kubernetes.io/projected/ff9d33cf-b02a-48b2-a8d7-4f3b38330369-kube-api-access-m6zx5\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:08.486372 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.486338    2536 generic.go:356] "Generic (PLEG): container finished" podID="ff9d33cf-b02a-48b2-a8d7-4f3b38330369" containerID="b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7" exitCode=0
Jul 02 20:23:08.486585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.486383    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"
Jul 02 20:23:08.486585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.486382    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k" event={"ID":"ff9d33cf-b02a-48b2-a8d7-4f3b38330369","Type":"ContainerDied","Data":"b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7"}
Jul 02 20:23:08.486585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.486532    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k" event={"ID":"ff9d33cf-b02a-48b2-a8d7-4f3b38330369","Type":"ContainerDied","Data":"a0da59ed6641ff45702e7bea5c79ed2f3e9d440a66ba547fd91e8a54c4ccec0e"}
Jul 02 20:23:08.486585 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.486571    2536 scope.go:117] "RemoveContainer" containerID="b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7"
Jul 02 20:23:08.495769 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.495747    2536 scope.go:117] "RemoveContainer" containerID="b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7"
Jul 02 20:23:08.496068 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:23:08.496046    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7\": container with ID starting with b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7 not found: ID does not exist" containerID="b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7"
Jul 02 20:23:08.496162 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.496075    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7"} err="failed to get container status \"b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7\": rpc error: code = NotFound desc = could not find container \"b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7\": container with ID starting with b599a3e4ed4defef01c1bcf519aecc60e2fcd3e6858558eee5410290528fe4c7 not found: ID does not exist"
Jul 02 20:23:08.504022 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.504001    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"]
Jul 02 20:23:08.505770 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:08.505751    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-ffaf0090-75fd546f87-6gq7k"]
Jul 02 20:23:10.370501 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:10.370467    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9d33cf-b02a-48b2-a8d7-4f3b38330369" path="/var/lib/kubelet/pods/ff9d33cf-b02a-48b2-a8d7-4f3b38330369/volumes"
Jul 02 20:23:12.448060 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.448031    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:12.508828 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.508749    2536 generic.go:356] "Generic (PLEG): container finished" podID="8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" containerID="c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599" exitCode=0
Jul 02 20:23:12.508828 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.508794    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"
Jul 02 20:23:12.509064 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.508830    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f" event={"ID":"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd","Type":"ContainerDied","Data":"c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599"}
Jul 02 20:23:12.509064 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.508890    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f" event={"ID":"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd","Type":"ContainerDied","Data":"80b84930300a0a0a73318494136f706385a19142534e5869ff6889b2ca2ec1c4"}
Jul 02 20:23:12.509064 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.508919    2536 scope.go:117] "RemoveContainer" containerID="c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599"
Jul 02 20:23:12.517460 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.517441    2536 scope.go:117] "RemoveContainer" containerID="c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599"
Jul 02 20:23:12.517706 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:23:12.517686    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599\": container with ID starting with c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599 not found: ID does not exist" containerID="c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599"
Jul 02 20:23:12.517762 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.517722    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599"} err="failed to get container status \"c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599\": rpc error: code = NotFound desc = could not find container \"c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599\": container with ID starting with c18de03d53bddc81205f03f010ae1b83027731c9260a7b778fdf7f555ff43599 not found: ID does not exist"
Jul 02 20:23:12.538116 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538088    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istiod-ca-cert\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538233 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538124    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq58v\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-kube-api-access-hq58v\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538233 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538160    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-certs\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538233 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538176    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-token\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538233 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538214    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-podinfo\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538470 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538243    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-data\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538470 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538269    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-credential-socket\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538470 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538381    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-envoy\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538470 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538417    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-socket\") pod \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\" (UID: \"8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd\") "
Jul 02 20:23:12.538659 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538540    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:23:12.538659 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538643    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:23:12.538767 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538645    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:23:12.538767 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538729    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:23:12.538871 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538817    2536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.538871 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538839    2536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istiod-ca-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.538871 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538854    2536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-workload-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.538871 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.538867    2536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-credential-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.539617 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.539592    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:23:12.540617 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.540582    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 20:23:12.540737 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.540687    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-kube-api-access-hq58v" (OuterVolumeSpecName: "kube-api-access-hq58v") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "kube-api-access-hq58v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:23:12.540783 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.540764    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:23:12.540985 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.540971    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" (UID: "8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:23:12.639692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.639658    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq58v\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-kube-api-access-hq58v\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.639692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.639687    2536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.639692 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.639698    2536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-podinfo\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.639926 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.639706    2536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-data\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.639926 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.639714    2536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd-istio-envoy\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:23:12.831135 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.831095    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"]
Jul 02 20:23:12.835589 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:12.835562    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ffaf0090-openshift-default-69fb6dc5f9-cns5f"]
Jul 02 20:23:14.368791 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:23:14.368755    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" path="/var/lib/kubelet/pods/8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd/volumes"
Jul 02 20:24:48.790102 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.790067    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"]
Jul 02 20:24:48.790580 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.790443    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9d33cf-b02a-48b2-a8d7-4f3b38330369" containerName="maas-api"
Jul 02 20:24:48.790580 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.790454    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9d33cf-b02a-48b2-a8d7-4f3b38330369" containerName="maas-api"
Jul 02 20:24:48.790580 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.790473    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" containerName="istio-proxy"
Jul 02 20:24:48.790580 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.790478    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" containerName="istio-proxy"
Jul 02 20:24:48.790580 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.790530    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9d33cf-b02a-48b2-a8d7-4f3b38330369" containerName="maas-api"
Jul 02 20:24:48.790580 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.790540    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b3e1c44-e7cd-41ce-8667-952e5a2ca1cd" containerName="istio-proxy"
Jul 02 20:24:48.793484 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.793469    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.795924 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.795904    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4f039091-openshift-default-dockercfg-74pdx\""
Jul 02 20:24:48.813648 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.813604    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"]
Jul 02 20:24:48.893046 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893015    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-certs\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893046 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893050    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-podinfo\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893265 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893114    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-socket\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893265 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893132    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-envoy\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893265 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893216    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-data\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893265 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893240    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-token\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893265 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893260    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-credential-socket\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893459 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893306    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525vr\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-kube-api-access-525vr\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.893459 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.893341    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istiod-ca-cert\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994165 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994138    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istiod-ca-cert\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994165 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994170    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-certs\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994440 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994187    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-podinfo\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994440 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994323    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-socket\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994440 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994357    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-envoy\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994440 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994424    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-data\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994651 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994463    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-token\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994651 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994491    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-credential-socket\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994651 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994521    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525vr\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-kube-api-access-525vr\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994644    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-certs\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.994998 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.994933    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istiod-ca-cert\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.995168 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.995023    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-data\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.995168 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.995039    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-socket\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.995168 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.995062    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-credential-socket\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.996841 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.996810    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-envoy\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:48.996966 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:48.996950    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-podinfo\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:49.002384 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.002361    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-token\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:49.002474 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.002422    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-525vr\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-kube-api-access-525vr\") pod \"e2e-mt-4f039091-openshift-default-57f965556c-hkhd6\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") " pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:49.103381 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.103277    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:49.256732 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.256624    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"]
Jul 02 20:24:49.259026 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:24:49.258999    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bcd503b_b306_445f_9f3f_ef8f1ffb1857.slice/crio-f51a2265558dd44fe9dcc5e91c4fcc6a838b8686053804d68ba004db01dc9ae7 WatchSource:0}: Error finding container f51a2265558dd44fe9dcc5e91c4fcc6a838b8686053804d68ba004db01dc9ae7: Status 404 returned error can't find the container with id f51a2265558dd44fe9dcc5e91c4fcc6a838b8686053804d68ba004db01dc9ae7
Jul 02 20:24:49.261196 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.261156    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:24:49.261276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.261226    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:24:49.261276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.261257    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:24:49.962264 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.962185    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" event={"ID":"6bcd503b-b306-445f-9f3f-ef8f1ffb1857","Type":"ContainerStarted","Data":"bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d"}
Jul 02 20:24:49.962264 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.962239    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" event={"ID":"6bcd503b-b306-445f-9f3f-ef8f1ffb1857","Type":"ContainerStarted","Data":"f51a2265558dd44fe9dcc5e91c4fcc6a838b8686053804d68ba004db01dc9ae7"}
Jul 02 20:24:49.982982 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:49.982933    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" podStartSLOduration=1.982919385 podStartE2EDuration="1.982919385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:24:49.981766335 +0000 UTC m=+1702.129535114" watchObservedRunningTime="2026-07-02 20:24:49.982919385 +0000 UTC m=+1702.130688163"
Jul 02 20:24:50.104111 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:50.104076    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:50.108517 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:50.108491    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:50.967314 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:50.967260    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:50.968120 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:50.968102    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:24:52.295843 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.295808    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"]
Jul 02 20:24:52.300091 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.300069    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.302511 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.302489    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4f039091-serving-cert\""
Jul 02 20:24:52.307127 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.307104    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"]
Jul 02 20:24:52.426955 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.426921    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-openshift-service-ca\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.427148 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.426987    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-maas-api-tls\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.427148 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.427056    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj4fc\" (UniqueName: \"kubernetes.io/projected/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-kube-api-access-vj4fc\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.528370 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.528338    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-maas-api-tls\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.528563 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.528379    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj4fc\" (UniqueName: \"kubernetes.io/projected/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-kube-api-access-vj4fc\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.528563 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.528491    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-openshift-service-ca\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.529151 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.529131    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-openshift-service-ca\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.531127 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.531105    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-maas-api-tls\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.541409 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.541383    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj4fc\" (UniqueName: \"kubernetes.io/projected/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-kube-api-access-vj4fc\") pod \"maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") " pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.612709 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.612642    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:52.758493 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.758414    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"]
Jul 02 20:24:52.760745 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:24:52.760715    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a16dfeb_3dac_4670_ba0c_88ca8cf6f168.slice/crio-bb0ff3fa9c75e6e5527a569296deb8f6b4835e9998c5dd95ef8d8fc67d92ef42 WatchSource:0}: Error finding container bb0ff3fa9c75e6e5527a569296deb8f6b4835e9998c5dd95ef8d8fc67d92ef42: Status 404 returned error can't find the container with id bb0ff3fa9c75e6e5527a569296deb8f6b4835e9998c5dd95ef8d8fc67d92ef42
Jul 02 20:24:52.978141 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:52.978106    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9" event={"ID":"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168","Type":"ContainerStarted","Data":"bb0ff3fa9c75e6e5527a569296deb8f6b4835e9998c5dd95ef8d8fc67d92ef42"}
Jul 02 20:24:53.984144 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:53.984106    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9" event={"ID":"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168","Type":"ContainerStarted","Data":"052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a"}
Jul 02 20:24:53.984586 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:53.984247    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:24:54.002051 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:54.002004    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9" podStartSLOduration=1.569421897 podStartE2EDuration="2.00198997s" totalImagesPullingTime="432.568073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:24:54.000195677 +0000 UTC m=+1706.147964493" watchObservedRunningTime="2026-07-02 20:24:54.00198997 +0000 UTC m=+1706.149758748"
Jul 02 20:24:59.995263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:24:59.995235    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:25:04.034614 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.034533    2536 generic.go:356] "Generic (PLEG): container finished" podID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerID="bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d" exitCode=0
Jul 02 20:25:04.035024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.034608    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" event={"ID":"6bcd503b-b306-445f-9f3f-ef8f1ffb1857","Type":"ContainerDied","Data":"bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d"}
Jul 02 20:25:04.035024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.034957    2536 scope.go:117] "RemoveContainer" containerID="bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d"
Jul 02 20:25:04.035945 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.035920    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:04.036030 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.035983    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:04.036030 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.036014    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:04.200888 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.200856    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"]
Jul 02 20:25:04.201133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.201084    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9" podUID="7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" containerName="maas-api" containerID="cri-o://052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a" gracePeriod=30
Jul 02 20:25:04.470495 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.470465    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:25:04.536707 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.536674    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-openshift-service-ca\") pod \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") "
Jul 02 20:25:04.536892 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.536724    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-maas-api-tls\") pod \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") "
Jul 02 20:25:04.536892 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.536811    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj4fc\" (UniqueName: \"kubernetes.io/projected/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-kube-api-access-vj4fc\") pod \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\" (UID: \"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168\") "
Jul 02 20:25:04.537074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.537049    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" (UID: "7a16dfeb-3dac-4670-ba0c-88ca8cf6f168"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:25:04.538916 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.538884    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" (UID: "7a16dfeb-3dac-4670-ba0c-88ca8cf6f168"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:25:04.538993 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.538931    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-kube-api-access-vj4fc" (OuterVolumeSpecName: "kube-api-access-vj4fc") pod "7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" (UID: "7a16dfeb-3dac-4670-ba0c-88ca8cf6f168"). InnerVolumeSpecName "kube-api-access-vj4fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:25:04.637563 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.637542    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj4fc\" (UniqueName: \"kubernetes.io/projected/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-kube-api-access-vj4fc\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:04.637563 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.637564    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:04.637707 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:04.637574    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:05.040174 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.040084    2536 generic.go:356] "Generic (PLEG): container finished" podID="7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" containerID="052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a" exitCode=0
Jul 02 20:25:05.040174 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.040136    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"
Jul 02 20:25:05.040726 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.040175    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9" event={"ID":"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168","Type":"ContainerDied","Data":"052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a"}
Jul 02 20:25:05.040726 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.040230    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9" event={"ID":"7a16dfeb-3dac-4670-ba0c-88ca8cf6f168","Type":"ContainerDied","Data":"bb0ff3fa9c75e6e5527a569296deb8f6b4835e9998c5dd95ef8d8fc67d92ef42"}
Jul 02 20:25:05.040726 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.040259    2536 scope.go:117] "RemoveContainer" containerID="052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a"
Jul 02 20:25:05.042410 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.042374    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" event={"ID":"6bcd503b-b306-445f-9f3f-ef8f1ffb1857","Type":"ContainerStarted","Data":"ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9"}
Jul 02 20:25:05.050441 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.050417    2536 scope.go:117] "RemoveContainer" containerID="052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a"
Jul 02 20:25:05.050729 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:25:05.050698    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a\": container with ID starting with 052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a not found: ID does not exist" containerID="052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a"
Jul 02 20:25:05.050827 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.050741    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a"} err="failed to get container status \"052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a\": rpc error: code = NotFound desc = could not find container \"052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a\": container with ID starting with 052008b5056f0a1d6f0921dc5a622c54afc9ec90d4a38117496a9d2d6945e47a not found: ID does not exist"
Jul 02 20:25:05.078516 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.078485    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"]
Jul 02 20:25:05.081561 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.081534    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-4f039091-74755cfc7f-7wdc9"]
Jul 02 20:25:05.103967 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.103946    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:25:05.105117 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.105094    2536 patch_prober.go:28] interesting pod/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 02 20:25:05.105218 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.105144    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 02 20:25:05.968034 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:05.967992    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:25:06.104386 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:06.104351    2536 patch_prober.go:28] interesting pod/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 02 20:25:06.104780 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:06.104405    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 02 20:25:06.372702 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:06.372609    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" path="/var/lib/kubelet/pods/7a16dfeb-3dac-4670-ba0c-88ca8cf6f168/volumes"
Jul 02 20:25:07.104099 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:07.104063    2536 patch_prober.go:28] interesting pod/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 02 20:25:07.104249 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:07.104124    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 02 20:25:08.104303 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:08.104268    2536 patch_prober.go:28] interesting pod/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 02 20:25:08.104706 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:08.104330    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 02 20:25:09.107332 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:09.107303    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:25:09.108310 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:09.108276    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:25:10.390841 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:10.390810    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"]
Jul 02 20:25:11.077837 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:11.077800    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy" containerID="cri-o://ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9" gracePeriod=30
Jul 02 20:25:16.332522 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.332490    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:25:16.449847 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.449815    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-podinfo\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450018 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.449877    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-525vr\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-kube-api-access-525vr\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450018 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.449899    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-credential-socket\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450018 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450000    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-envoy\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450142 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450069    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-token\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450142 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450104    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-socket\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450142 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450128    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istiod-ca-cert\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450270 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450145    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:25:16.450270 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450181    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-data\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450270 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450217    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-certs\") pod \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\" (UID: \"6bcd503b-b306-445f-9f3f-ef8f1ffb1857\") "
Jul 02 20:25:16.450453 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450394    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:25:16.450812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450555    2536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.450812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450578    2536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-credential-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.450812 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.450608    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:25:16.451207 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.451076    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:25:16.451467 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.451441    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:25:16.452408 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.452382    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:25:16.452509 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.452405    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 20:25:16.452509 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.452414    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:25:16.452584 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.452534    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-kube-api-access-525vr" (OuterVolumeSpecName: "kube-api-access-525vr") pod "6bcd503b-b306-445f-9f3f-ef8f1ffb1857" (UID: "6bcd503b-b306-445f-9f3f-ef8f1ffb1857"). InnerVolumeSpecName "kube-api-access-525vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:25:16.551886 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.551861    2536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-podinfo\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.551886 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.551892    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-525vr\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-kube-api-access-525vr\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.552074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.551904    2536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-envoy\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.552074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.551916    2536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.552074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.551928    2536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istiod-ca-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.552074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.551935    2536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-istio-data\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:16.552074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:16.551943    2536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bcd503b-b306-445f-9f3f-ef8f1ffb1857-workload-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:25:17.113876 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.113821    2536 generic.go:356] "Generic (PLEG): container finished" podID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerID="ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9" exitCode=0
Jul 02 20:25:17.113876 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.113875    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" event={"ID":"6bcd503b-b306-445f-9f3f-ef8f1ffb1857","Type":"ContainerDied","Data":"ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9"}
Jul 02 20:25:17.114101 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.113888    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"
Jul 02 20:25:17.114101 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.113936    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6" event={"ID":"6bcd503b-b306-445f-9f3f-ef8f1ffb1857","Type":"ContainerDied","Data":"f51a2265558dd44fe9dcc5e91c4fcc6a838b8686053804d68ba004db01dc9ae7"}
Jul 02 20:25:17.114101 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.113968    2536 scope.go:117] "RemoveContainer" containerID="ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9"
Jul 02 20:25:17.123399 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.123372    2536 scope.go:117] "RemoveContainer" containerID="bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d"
Jul 02 20:25:17.131260 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.131240    2536 scope.go:117] "RemoveContainer" containerID="ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9"
Jul 02 20:25:17.131603 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:25:17.131565    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9\": container with ID starting with ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9 not found: ID does not exist" containerID="ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9"
Jul 02 20:25:17.131708 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.131616    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9"} err="failed to get container status \"ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9\": rpc error: code = NotFound desc = could not find container \"ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9\": container with ID starting with ca5f6a5912d0d91c703d4e85ecbbb2fddf1fbd277868014ab7e0c4495728a3e9 not found: ID does not exist"
Jul 02 20:25:17.131708 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.131640    2536 scope.go:117] "RemoveContainer" containerID="bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d"
Jul 02 20:25:17.131995 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:25:17.131968    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d\": container with ID starting with bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d not found: ID does not exist" containerID="bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d"
Jul 02 20:25:17.132048 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.132007    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d"} err="failed to get container status \"bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d\": rpc error: code = NotFound desc = could not find container \"bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d\": container with ID starting with bbd5bed1ca8e3bb104fd4fc481977938e8847de3ff0c30b8b6dd824fab74778d not found: ID does not exist"
Jul 02 20:25:17.144688 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.144665    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"]
Jul 02 20:25:17.146470 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:17.146450    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4f039091-openshift-default-57f965556c-hkhd6"]
Jul 02 20:25:18.368727 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:18.368694    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" path="/var/lib/kubelet/pods/6bcd503b-b306-445f-9f3f-ef8f1ffb1857/volumes"
Jul 02 20:25:30.342374 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342343    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"]
Jul 02 20:25:30.342799 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342721    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" containerName="maas-api"
Jul 02 20:25:30.342799 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342731    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" containerName="maas-api"
Jul 02 20:25:30.342799 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342745    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy"
Jul 02 20:25:30.342799 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342751    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy"
Jul 02 20:25:30.342799 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342780    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy"
Jul 02 20:25:30.342799 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342785    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy"
Jul 02 20:25:30.342988 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342838    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a16dfeb-3dac-4670-ba0c-88ca8cf6f168" containerName="maas-api"
Jul 02 20:25:30.342988 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342849    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy"
Jul 02 20:25:30.342988 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.342855    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bcd503b-b306-445f-9f3f-ef8f1ffb1857" containerName="istio-proxy"
Jul 02 20:25:30.346880 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.346857    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.349561 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.349539    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-eb7f1a51-openshift-default-dockercfg-cdb79\""
Jul 02 20:25:30.359304 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.359266    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"]
Jul 02 20:25:30.467321 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467271    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-socket\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467497 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467338    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-credential-socket\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467497 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467381    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-certs\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467497 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467398    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-envoy\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467497 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467422    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-data\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467497 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467477    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvtdv\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-kube-api-access-mvtdv\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467758 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467552    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-podinfo\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467758 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467615    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-token\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.467758 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.467651    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882443b4-6aa0-4bce-bd08-2c2185d7a312-istiod-ca-cert\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568306 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568258    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-socket\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568446 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568316    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-credential-socket\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568446 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568368    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-certs\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568446 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568395    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-envoy\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568446 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568419    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-data\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568446 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568439    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvtdv\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-kube-api-access-mvtdv\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568698 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568474    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-podinfo\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568698 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568524    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-token\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568698 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568558    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882443b4-6aa0-4bce-bd08-2c2185d7a312-istiod-ca-cert\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568840 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568693    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-socket\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568840 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568747    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-credential-socket\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.568985 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568949    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-certs\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.569073 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.568952    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-data\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.569464 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.569444    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882443b4-6aa0-4bce-bd08-2c2185d7a312-istiod-ca-cert\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.570876 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.570850    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-envoy\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.571069 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.571052    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-podinfo\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.578424 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.578390    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvtdv\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-kube-api-access-mvtdv\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.579181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.579166    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-token\") pod \"e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") " pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.657542 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.657507    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:30.813020 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.812990    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"]
Jul 02 20:25:30.814399 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:25:30.814372    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882443b4_6aa0_4bce_bd08_2c2185d7a312.slice/crio-d0d9d86db84b8e249a426446240f19bad6a03de795ee0e94b418f5d336aa436f WatchSource:0}: Error finding container d0d9d86db84b8e249a426446240f19bad6a03de795ee0e94b418f5d336aa436f: Status 404 returned error can't find the container with id d0d9d86db84b8e249a426446240f19bad6a03de795ee0e94b418f5d336aa436f
Jul 02 20:25:30.816397 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.816368    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:30.816464 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.816429    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:30.816464 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:30.816457    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:31.181927 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:31.181892    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" event={"ID":"882443b4-6aa0-4bce-bd08-2c2185d7a312","Type":"ContainerStarted","Data":"d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614"}
Jul 02 20:25:31.182095 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:31.181953    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" event={"ID":"882443b4-6aa0-4bce-bd08-2c2185d7a312","Type":"ContainerStarted","Data":"d0d9d86db84b8e249a426446240f19bad6a03de795ee0e94b418f5d336aa436f"}
Jul 02 20:25:31.202253 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:31.202202    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" podStartSLOduration=1.202185392 podStartE2EDuration="1.202185392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:25:31.201664206 +0000 UTC m=+1743.349432985" watchObservedRunningTime="2026-07-02 20:25:31.202185392 +0000 UTC m=+1743.349954171"
Jul 02 20:25:31.658228 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:31.658198    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:31.663867 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:31.663845    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:32.186947 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:32.186914    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:32.187836 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:32.187820    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:33.514742 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.514713    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"]
Jul 02 20:25:33.518134 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.518118    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.520482 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.520460    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-eb7f1a51-serving-cert\""
Jul 02 20:25:33.529021 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.528999    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"]
Jul 02 20:25:33.593443 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.593408    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a82ec3-6741-43d1-9f71-3f5ea602edaf-openshift-service-ca\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.593649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.593449    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trkdl\" (UniqueName: \"kubernetes.io/projected/25a82ec3-6741-43d1-9f71-3f5ea602edaf-kube-api-access-trkdl\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.593649 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.593499    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a82ec3-6741-43d1-9f71-3f5ea602edaf-maas-api-tls\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.694051 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.694017    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a82ec3-6741-43d1-9f71-3f5ea602edaf-maas-api-tls\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.694255 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.694128    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a82ec3-6741-43d1-9f71-3f5ea602edaf-openshift-service-ca\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.694255 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.694155    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trkdl\" (UniqueName: \"kubernetes.io/projected/25a82ec3-6741-43d1-9f71-3f5ea602edaf-kube-api-access-trkdl\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.694755 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.694733    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a82ec3-6741-43d1-9f71-3f5ea602edaf-openshift-service-ca\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.696593 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.696573    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a82ec3-6741-43d1-9f71-3f5ea602edaf-maas-api-tls\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.704580 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.704558    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trkdl\" (UniqueName: \"kubernetes.io/projected/25a82ec3-6741-43d1-9f71-3f5ea602edaf-kube-api-access-trkdl\") pod \"maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") " pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.828874 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.828803    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:33.963108 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:33.963085    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"]
Jul 02 20:25:33.964584 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:25:33.964557    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a82ec3_6741_43d1_9f71_3f5ea602edaf.slice/crio-e40885ca0ee931d5a3f1b881d1ed40273c6d395f89b1f1b126346cb9228d8a17 WatchSource:0}: Error finding container e40885ca0ee931d5a3f1b881d1ed40273c6d395f89b1f1b126346cb9228d8a17: Status 404 returned error can't find the container with id e40885ca0ee931d5a3f1b881d1ed40273c6d395f89b1f1b126346cb9228d8a17
Jul 02 20:25:34.197850 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:34.197818    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz" event={"ID":"25a82ec3-6741-43d1-9f71-3f5ea602edaf","Type":"ContainerStarted","Data":"e40885ca0ee931d5a3f1b881d1ed40273c6d395f89b1f1b126346cb9228d8a17"}
Jul 02 20:25:35.203943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:35.203909    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz" event={"ID":"25a82ec3-6741-43d1-9f71-3f5ea602edaf","Type":"ContainerStarted","Data":"79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76"}
Jul 02 20:25:35.204340 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:35.203963    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:35.225520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:35.225465    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz" podStartSLOduration=1.8331235879999999 podStartE2EDuration="2.22545081s" totalImagesPullingTime="392.327222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:25:35.224868883 +0000 UTC m=+1747.372637662" watchObservedRunningTime="2026-07-02 20:25:35.22545081 +0000 UTC m=+1747.373219617"
Jul 02 20:25:41.216887 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:41.216857    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:25:44.251444 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:44.251366    2536 generic.go:356] "Generic (PLEG): container finished" podID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerID="d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614" exitCode=0
Jul 02 20:25:44.251872 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:44.251442    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" event={"ID":"882443b4-6aa0-4bce-bd08-2c2185d7a312","Type":"ContainerDied","Data":"d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614"}
Jul 02 20:25:44.251872 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:44.251771    2536 scope.go:117] "RemoveContainer" containerID="d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614"
Jul 02 20:25:44.252848 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:44.252817    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:44.252954 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:44.252904    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:44.253011 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:44.252957    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:25:45.262133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:45.262084    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" event={"ID":"882443b4-6aa0-4bce-bd08-2c2185d7a312","Type":"ContainerStarted","Data":"f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9"}
Jul 02 20:25:45.658236 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:45.658144    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:45.659677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:45.659649    2536 patch_prober.go:28] interesting pod/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 02 20:25:45.659893 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:45.659868    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 02 20:25:46.664309 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:46.661635    2536 patch_prober.go:28] interesting pod/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 02 20:25:46.664309 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:46.661705    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 02 20:25:47.187156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:47.187114    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:47.658701 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:47.658669    2536 patch_prober.go:28] interesting pod/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 02 20:25:47.658976 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:47.658947    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 02 20:25:48.658924 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:48.658885    2536 patch_prober.go:28] interesting pod/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 02 20:25:48.659555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:48.659525    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 02 20:25:49.662520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:49.662490    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:25:49.663592 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:25:49.663574    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:26:21.362569 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.362471    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"]
Jul 02 20:26:21.363154 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.362744    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz" podUID="25a82ec3-6741-43d1-9f71-3f5ea602edaf" containerName="maas-api" containerID="cri-o://79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76" gracePeriod=30
Jul 02 20:26:21.608793 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.608769    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:26:21.738183 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.738154    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a82ec3-6741-43d1-9f71-3f5ea602edaf-openshift-service-ca\") pod \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") "
Jul 02 20:26:21.738401 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.738261    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a82ec3-6741-43d1-9f71-3f5ea602edaf-maas-api-tls\") pod \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") "
Jul 02 20:26:21.738401 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.738315    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trkdl\" (UniqueName: \"kubernetes.io/projected/25a82ec3-6741-43d1-9f71-3f5ea602edaf-kube-api-access-trkdl\") pod \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\" (UID: \"25a82ec3-6741-43d1-9f71-3f5ea602edaf\") "
Jul 02 20:26:21.738640 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.738620    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a82ec3-6741-43d1-9f71-3f5ea602edaf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25a82ec3-6741-43d1-9f71-3f5ea602edaf" (UID: "25a82ec3-6741-43d1-9f71-3f5ea602edaf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:26:21.740429 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.740403    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a82ec3-6741-43d1-9f71-3f5ea602edaf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25a82ec3-6741-43d1-9f71-3f5ea602edaf" (UID: "25a82ec3-6741-43d1-9f71-3f5ea602edaf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:26:21.740535 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.740461    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a82ec3-6741-43d1-9f71-3f5ea602edaf-kube-api-access-trkdl" (OuterVolumeSpecName: "kube-api-access-trkdl") pod "25a82ec3-6741-43d1-9f71-3f5ea602edaf" (UID: "25a82ec3-6741-43d1-9f71-3f5ea602edaf"). InnerVolumeSpecName "kube-api-access-trkdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:26:21.839343 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.839313    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25a82ec3-6741-43d1-9f71-3f5ea602edaf-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:26:21.839343 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.839338    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25a82ec3-6741-43d1-9f71-3f5ea602edaf-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:26:21.839343 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:21.839348    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trkdl\" (UniqueName: \"kubernetes.io/projected/25a82ec3-6741-43d1-9f71-3f5ea602edaf-kube-api-access-trkdl\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:26:22.436335 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.436304    2536 generic.go:356] "Generic (PLEG): container finished" podID="25a82ec3-6741-43d1-9f71-3f5ea602edaf" containerID="79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76" exitCode=0
Jul 02 20:26:22.436699 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.436352    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"
Jul 02 20:26:22.436699 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.436379    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz" event={"ID":"25a82ec3-6741-43d1-9f71-3f5ea602edaf","Type":"ContainerDied","Data":"79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76"}
Jul 02 20:26:22.436699 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.436425    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz" event={"ID":"25a82ec3-6741-43d1-9f71-3f5ea602edaf","Type":"ContainerDied","Data":"e40885ca0ee931d5a3f1b881d1ed40273c6d395f89b1f1b126346cb9228d8a17"}
Jul 02 20:26:22.436699 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.436446    2536 scope.go:117] "RemoveContainer" containerID="79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76"
Jul 02 20:26:22.444599 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.444580    2536 scope.go:117] "RemoveContainer" containerID="79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76"
Jul 02 20:26:22.444845 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:26:22.444822    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76\": container with ID starting with 79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76 not found: ID does not exist" containerID="79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76"
Jul 02 20:26:22.444911 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.444858    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76"} err="failed to get container status \"79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76\": rpc error: code = NotFound desc = could not find container \"79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76\": container with ID starting with 79d7848cebede8a547ec8e4db2e672e2b5db598c67a455dab464f0c25fd1ad76 not found: ID does not exist"
Jul 02 20:26:22.452133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.452110    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"]
Jul 02 20:26:22.455115 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:22.455097    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-eb7f1a51-67b6dfffc-gxvzz"]
Jul 02 20:26:24.368914 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:26:24.368883    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25a82ec3-6741-43d1-9f71-3f5ea602edaf" path="/var/lib/kubelet/pods/25a82ec3-6741-43d1-9f71-3f5ea602edaf/volumes"
Jul 02 20:27:37.906074 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:37.906042    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"]
Jul 02 20:27:37.906639 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:37.906267    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy" containerID="cri-o://f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9" gracePeriod=30
Jul 02 20:27:43.164690 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.164532    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:27:43.236887 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.236855    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-data\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.236887 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.236894    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-socket\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237230 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.236919    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-token\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237230 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.236941    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-envoy\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237230 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.236973    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882443b4-6aa0-4bce-bd08-2c2185d7a312-istiod-ca-cert\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237230 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237029    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-podinfo\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237230 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237066    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-credential-socket\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237240    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvtdv\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-kube-api-access-mvtdv\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237332    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-certs\") pod \"882443b4-6aa0-4bce-bd08-2c2185d7a312\" (UID: \"882443b4-6aa0-4bce-bd08-2c2185d7a312\") "
Jul 02 20:27:43.237520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237233    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:27:43.237520 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237452    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:27:43.237712 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237641    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:27:43.237712 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237661    2536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-credential-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.237712 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237680    2536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.237982 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.237955    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/882443b4-6aa0-4bce-bd08-2c2185d7a312-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:27:43.238094 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.238063    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-data" (OuterVolumeSpecName: "istio-data") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:27:43.239394 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.239363    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-kube-api-access-mvtdv" (OuterVolumeSpecName: "kube-api-access-mvtdv") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "kube-api-access-mvtdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:27:43.239527 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.239394    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-token" (OuterVolumeSpecName: "istio-token") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:27:43.239527 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.239428    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:27:43.239744 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.239721    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "882443b4-6aa0-4bce-bd08-2c2185d7a312" (UID: "882443b4-6aa0-4bce-bd08-2c2185d7a312"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 20:27:43.338910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.338873    2536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-data\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.338910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.338901    2536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.338910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.338914    2536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-envoy\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.339163 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.338925    2536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/882443b4-6aa0-4bce-bd08-2c2185d7a312-istiod-ca-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.339163 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.338937    2536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/882443b4-6aa0-4bce-bd08-2c2185d7a312-istio-podinfo\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.339163 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.338948    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvtdv\" (UniqueName: \"kubernetes.io/projected/882443b4-6aa0-4bce-bd08-2c2185d7a312-kube-api-access-mvtdv\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.339163 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.338962    2536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/882443b4-6aa0-4bce-bd08-2c2185d7a312-workload-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:27:43.815551 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.815510    2536 generic.go:356] "Generic (PLEG): container finished" podID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerID="f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9" exitCode=0
Jul 02 20:27:43.815551 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.815541    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" event={"ID":"882443b4-6aa0-4bce-bd08-2c2185d7a312","Type":"ContainerDied","Data":"f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9"}
Jul 02 20:27:43.815777 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.815557    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"
Jul 02 20:27:43.815777 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.815600    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m" event={"ID":"882443b4-6aa0-4bce-bd08-2c2185d7a312","Type":"ContainerDied","Data":"d0d9d86db84b8e249a426446240f19bad6a03de795ee0e94b418f5d336aa436f"}
Jul 02 20:27:43.815777 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.815627    2536 scope.go:117] "RemoveContainer" containerID="f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9"
Jul 02 20:27:43.825211 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.825191    2536 scope.go:117] "RemoveContainer" containerID="d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614"
Jul 02 20:27:43.834079 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.834061    2536 scope.go:117] "RemoveContainer" containerID="f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9"
Jul 02 20:27:43.834364 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:27:43.834346    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9\": container with ID starting with f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9 not found: ID does not exist" containerID="f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9"
Jul 02 20:27:43.834415 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.834374    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9"} err="failed to get container status \"f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9\": rpc error: code = NotFound desc = could not find container \"f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9\": container with ID starting with f2b829e3afbccfea84a33dad7c5e1189ee27f39ab50f87e9993d348e61da38d9 not found: ID does not exist"
Jul 02 20:27:43.834415 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.834392    2536 scope.go:117] "RemoveContainer" containerID="d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614"
Jul 02 20:27:43.834650 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:27:43.834629    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614\": container with ID starting with d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614 not found: ID does not exist" containerID="d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614"
Jul 02 20:27:43.834688 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.834655    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614"} err="failed to get container status \"d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614\": rpc error: code = NotFound desc = could not find container \"d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614\": container with ID starting with d88d6851c365ff7334677de4d8c28ad974b1abaa88b80268cd4f43012bb77614 not found: ID does not exist"
Jul 02 20:27:43.838250 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.838226    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"]
Jul 02 20:27:43.844663 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:43.844640    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-eb7f1a51-openshift-default-6c97cc76bc-hg44m"]
Jul 02 20:27:44.368921 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:27:44.368888    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" path="/var/lib/kubelet/pods/882443b4-6aa0-4bce-bd08-2c2185d7a312/volumes"
Jul 02 20:28:32.463554 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.463524    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"]
Jul 02 20:28:32.464091 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.464072    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy"
Jul 02 20:28:32.464177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.464092    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy"
Jul 02 20:28:32.464177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.464110    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a82ec3-6741-43d1-9f71-3f5ea602edaf" containerName="maas-api"
Jul 02 20:28:32.464177 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.464118    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a82ec3-6741-43d1-9f71-3f5ea602edaf" containerName="maas-api"
Jul 02 20:28:32.464350 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.464236    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="25a82ec3-6741-43d1-9f71-3f5ea602edaf" containerName="maas-api"
Jul 02 20:28:32.464350 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.464256    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy"
Jul 02 20:28:32.464350 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.464265    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy"
Jul 02 20:28:32.467349 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.467330    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.469625 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.469608    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-55ba03-openshift-default-dockercfg-cv269\""
Jul 02 20:28:32.477250 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.477230    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"]
Jul 02 20:28:32.567602 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567567    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmpq5\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-kube-api-access-dmpq5\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.567787 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567615    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-certs\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.567787 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567685    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-envoy\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.567787 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567716    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-token\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.567787 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567733    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff10733-bd0f-4cd9-a672-952abf18af25-istiod-ca-cert\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.567787 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567755    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-socket\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.567787 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567781    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-credential-socket\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.568133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567840    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-data\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.568133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.567879    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-podinfo\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668335 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668279    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-token\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668335 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668339    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff10733-bd0f-4cd9-a672-952abf18af25-istiod-ca-cert\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668360    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-socket\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668379    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-credential-socket\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668409    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-data\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668445    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-podinfo\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668475    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmpq5\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-kube-api-access-dmpq5\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668545 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668515    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-certs\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668845 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668574    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-envoy\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668904 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668857    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-credential-socket\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.668964 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668934    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-data\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.669017 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.668962    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-socket\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.669056 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.669022    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-certs\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.669301 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.669267    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff10733-bd0f-4cd9-a672-952abf18af25-istiod-ca-cert\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.671021 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.671001    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-envoy\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.671111 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.671060    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-podinfo\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.677312 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.677261    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-token\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.677661 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.677639    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmpq5\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-kube-api-access-dmpq5\") pod \"e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") " pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.777991 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.777921    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:32.915596 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.915573    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"]
Jul 02 20:28:32.917694 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:28:32.917669    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff10733_bd0f_4cd9_a672_952abf18af25.slice/crio-e501d0a301124baa5a05e6f654c78e5fb910723d251b2ca970bb8abf8bf62fd4 WatchSource:0}: Error finding container e501d0a301124baa5a05e6f654c78e5fb910723d251b2ca970bb8abf8bf62fd4: Status 404 returned error can't find the container with id e501d0a301124baa5a05e6f654c78e5fb910723d251b2ca970bb8abf8bf62fd4
Jul 02 20:28:32.919713 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.919680    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:28:32.919797 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.919740    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:28:32.919797 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:32.919767    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:28:33.044133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:33.044036    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" event={"ID":"4ff10733-bd0f-4cd9-a672-952abf18af25","Type":"ContainerStarted","Data":"218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6"}
Jul 02 20:28:33.044133 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:33.044100    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" event={"ID":"4ff10733-bd0f-4cd9-a672-952abf18af25","Type":"ContainerStarted","Data":"e501d0a301124baa5a05e6f654c78e5fb910723d251b2ca970bb8abf8bf62fd4"}
Jul 02 20:28:33.065015 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:33.064965    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" podStartSLOduration=1.064947703 podStartE2EDuration="1.064947703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:28:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:28:33.063893729 +0000 UTC m=+1925.211662556" watchObservedRunningTime="2026-07-02 20:28:33.064947703 +0000 UTC m=+1925.212716507"
Jul 02 20:28:33.778516 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:33.778488    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:33.783503 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:33.783479    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:34.049238 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:34.049142    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:34.049990 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:34.049972    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:35.372062 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.372018    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"]
Jul 02 20:28:35.373423 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.373401    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy"
Jul 02 20:28:35.373571 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.373560    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="882443b4-6aa0-4bce-bd08-2c2185d7a312" containerName="istio-proxy"
Jul 02 20:28:35.377863 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.377841    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.380557 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.380530    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-55ba03-serving-cert\""
Jul 02 20:28:35.382910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.382885    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"]
Jul 02 20:28:35.390650 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.390626    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4c6w\" (UniqueName: \"kubernetes.io/projected/277128bb-fe88-4a49-9190-44839e69ac6b-kube-api-access-v4c6w\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.390759 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.390668    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.390844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.390824    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277128bb-fe88-4a49-9190-44839e69ac6b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.492264 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.492215    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277128bb-fe88-4a49-9190-44839e69ac6b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.492479 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.492337    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4c6w\" (UniqueName: \"kubernetes.io/projected/277128bb-fe88-4a49-9190-44839e69ac6b-kube-api-access-v4c6w\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.492479 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.492381    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.492750 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:28:35.492526    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-55ba03-serving-cert: secret "maas-api-e2e-inf-a-55ba03-serving-cert" not found
Jul 02 20:28:35.492750 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:28:35.492591    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls podName:277128bb-fe88-4a49-9190-44839e69ac6b nodeName:}" failed. No retries permitted until 2026-07-02 20:28:35.992568749 +0000 UTC m=+1928.140337525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls") pod "maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw" (UID: "277128bb-fe88-4a49-9190-44839e69ac6b") : secret "maas-api-e2e-inf-a-55ba03-serving-cert" not found
Jul 02 20:28:35.493203 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.493180    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277128bb-fe88-4a49-9190-44839e69ac6b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.502227 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.502202    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4c6w\" (UniqueName: \"kubernetes.io/projected/277128bb-fe88-4a49-9190-44839e69ac6b-kube-api-access-v4c6w\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:35.997642 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:35.997608    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:36.000092 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:36.000070    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls\") pod \"maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") " pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:36.290513 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:36.290406    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:36.428701 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:36.428678    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"]
Jul 02 20:28:36.430615 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:28:36.430590    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277128bb_fe88_4a49_9190_44839e69ac6b.slice/crio-88d16107010466176de79ceb6546d83788b73fac31d452238fc0edae4daa73b7 WatchSource:0}: Error finding container 88d16107010466176de79ceb6546d83788b73fac31d452238fc0edae4daa73b7: Status 404 returned error can't find the container with id 88d16107010466176de79ceb6546d83788b73fac31d452238fc0edae4daa73b7
Jul 02 20:28:36.432334 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:36.432316    2536 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 20:28:37.064388 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:37.064301    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw" event={"ID":"277128bb-fe88-4a49-9190-44839e69ac6b","Type":"ContainerStarted","Data":"3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33"}
Jul 02 20:28:37.064645 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:37.064623    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw" event={"ID":"277128bb-fe88-4a49-9190-44839e69ac6b","Type":"ContainerStarted","Data":"88d16107010466176de79ceb6546d83788b73fac31d452238fc0edae4daa73b7"}
Jul 02 20:28:37.065017 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:37.064965    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:37.082754 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:37.082709    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw" podStartSLOduration=1.7353635920000001 podStartE2EDuration="2.082694418s" totalImagesPullingTime="347.330826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:28:37.081112149 +0000 UTC m=+1929.228880929" watchObservedRunningTime="2026-07-02 20:28:37.082694418 +0000 UTC m=+1929.230463196"
Jul 02 20:28:43.075677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:43.075645    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:28:47.117253 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.117219    2536 generic.go:356] "Generic (PLEG): container finished" podID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerID="218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6" exitCode=0
Jul 02 20:28:47.117686 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.117299    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" event={"ID":"4ff10733-bd0f-4cd9-a672-952abf18af25","Type":"ContainerDied","Data":"218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6"}
Jul 02 20:28:47.117730 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.117695    2536 scope.go:117] "RemoveContainer" containerID="218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6"
Jul 02 20:28:47.118710 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.118685    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:28:47.118765 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.118752    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:28:47.118798 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.118786    2536 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 02 20:28:47.419655 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.419618    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"]
Jul 02 20:28:47.423954 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.423928    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.427047 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.427020    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-fe96cd-serving-cert\""
Jul 02 20:28:47.441500 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.441469    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"]
Jul 02 20:28:47.502949 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.502907    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.502949 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.502951    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.503200 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.503078    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v25rb\" (UniqueName: \"kubernetes.io/projected/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-kube-api-access-v25rb\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.604366 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.604325    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.604366 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.604366    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.604638 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.604441    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v25rb\" (UniqueName: \"kubernetes.io/projected/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-kube-api-access-v25rb\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.604638 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:28:47.604503    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-fe96cd-serving-cert: secret "maas-api-e2e-inf-b-fe96cd-serving-cert" not found
Jul 02 20:28:47.604638 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:28:47.604601    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls podName:8c823cb7-d04b-4b58-b824-8a80fe91a8a2 nodeName:}" failed. No retries permitted until 2026-07-02 20:28:48.104578077 +0000 UTC m=+1940.252346834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls") pod "maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2" (UID: "8c823cb7-d04b-4b58-b824-8a80fe91a8a2") : secret "maas-api-e2e-inf-b-fe96cd-serving-cert" not found
Jul 02 20:28:47.605069 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.605050    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:47.614702 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:47.614669    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v25rb\" (UniqueName: \"kubernetes.io/projected/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-kube-api-access-v25rb\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:48.109149 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.109114    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:48.111849 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.111818    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls\") pod \"maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") " pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:48.125121 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.125087    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" event={"ID":"4ff10733-bd0f-4cd9-a672-952abf18af25","Type":"ContainerStarted","Data":"548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf"}
Jul 02 20:28:48.335738 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.335695    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:48.528666 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.528641    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"]
Jul 02 20:28:48.531246 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:28:48.531220    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c823cb7_d04b_4b58_b824_8a80fe91a8a2.slice/crio-45605540a1210f62d25fe2dc50ccff873efd13966fbd20e66815735785b43135 WatchSource:0}: Error finding container 45605540a1210f62d25fe2dc50ccff873efd13966fbd20e66815735785b43135: Status 404 returned error can't find the container with id 45605540a1210f62d25fe2dc50ccff873efd13966fbd20e66815735785b43135
Jul 02 20:28:48.778997 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.778904    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:48.780353 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.780322    2536 patch_prober.go:28] interesting pod/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 02 20:28:48.780485 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:48.780378    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 02 20:28:49.049565 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:49.049407    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:49.131643 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:49.131599    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2" event={"ID":"8c823cb7-d04b-4b58-b824-8a80fe91a8a2","Type":"ContainerStarted","Data":"73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba"}
Jul 02 20:28:49.132132 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:49.131671    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2" event={"ID":"8c823cb7-d04b-4b58-b824-8a80fe91a8a2","Type":"ContainerStarted","Data":"45605540a1210f62d25fe2dc50ccff873efd13966fbd20e66815735785b43135"}
Jul 02 20:28:49.132132 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:49.131856    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:28:49.154451 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:49.153823    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2" podStartSLOduration=1.7807727039999999 podStartE2EDuration="2.153799056s" totalImagesPullingTime="373.026352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:28:49.150122552 +0000 UTC m=+1941.297891334" watchObservedRunningTime="2026-07-02 20:28:49.153799056 +0000 UTC m=+1941.301567837"
Jul 02 20:28:49.778602 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:49.778557    2536 patch_prober.go:28] interesting pod/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 02 20:28:49.778809 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:49.778625    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 02 20:28:50.778449 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:50.778408    2536 patch_prober.go:28] interesting pod/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 02 20:28:50.778898 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:50.778470    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 02 20:28:51.778648 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:51.778617    2536 patch_prober.go:28] interesting pod/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 02 20:28:51.779043 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:51.778692    2536 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 02 20:28:52.782946 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:52.782917    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:52.783935 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:52.783913    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:28:55.143123 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:28:55.143088    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:30:07.723395 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:07.723358    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"]
Jul 02 20:30:07.723943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:07.723632    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw" podUID="277128bb-fe88-4a49-9190-44839e69ac6b" containerName="maas-api" containerID="cri-o://3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33" gracePeriod=30
Jul 02 20:30:07.977548 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:07.977492    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:30:08.060516 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.060483    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277128bb-fe88-4a49-9190-44839e69ac6b-openshift-service-ca\") pod \"277128bb-fe88-4a49-9190-44839e69ac6b\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") "
Jul 02 20:30:08.060677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.060590    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4c6w\" (UniqueName: \"kubernetes.io/projected/277128bb-fe88-4a49-9190-44839e69ac6b-kube-api-access-v4c6w\") pod \"277128bb-fe88-4a49-9190-44839e69ac6b\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") "
Jul 02 20:30:08.060677 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.060625    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls\") pod \"277128bb-fe88-4a49-9190-44839e69ac6b\" (UID: \"277128bb-fe88-4a49-9190-44839e69ac6b\") "
Jul 02 20:30:08.060904 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.060882    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/277128bb-fe88-4a49-9190-44839e69ac6b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "277128bb-fe88-4a49-9190-44839e69ac6b" (UID: "277128bb-fe88-4a49-9190-44839e69ac6b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:30:08.062797 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.062772    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "277128bb-fe88-4a49-9190-44839e69ac6b" (UID: "277128bb-fe88-4a49-9190-44839e69ac6b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:30:08.062881 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.062859    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/277128bb-fe88-4a49-9190-44839e69ac6b-kube-api-access-v4c6w" (OuterVolumeSpecName: "kube-api-access-v4c6w") pod "277128bb-fe88-4a49-9190-44839e69ac6b" (UID: "277128bb-fe88-4a49-9190-44839e69ac6b"). InnerVolumeSpecName "kube-api-access-v4c6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:30:08.162320 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.162267    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4c6w\" (UniqueName: \"kubernetes.io/projected/277128bb-fe88-4a49-9190-44839e69ac6b-kube-api-access-v4c6w\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:08.162320 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.162316    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/277128bb-fe88-4a49-9190-44839e69ac6b-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:08.162555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.162328    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/277128bb-fe88-4a49-9190-44839e69ac6b-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:08.529455 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.529421    2536 generic.go:356] "Generic (PLEG): container finished" podID="277128bb-fe88-4a49-9190-44839e69ac6b" containerID="3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33" exitCode=0
Jul 02 20:30:08.529612 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.529467    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"
Jul 02 20:30:08.529612 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.529496    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw" event={"ID":"277128bb-fe88-4a49-9190-44839e69ac6b","Type":"ContainerDied","Data":"3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33"}
Jul 02 20:30:08.529612 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.529541    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw" event={"ID":"277128bb-fe88-4a49-9190-44839e69ac6b","Type":"ContainerDied","Data":"88d16107010466176de79ceb6546d83788b73fac31d452238fc0edae4daa73b7"}
Jul 02 20:30:08.529612 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.529565    2536 scope.go:117] "RemoveContainer" containerID="3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33"
Jul 02 20:30:08.538618 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.538600    2536 scope.go:117] "RemoveContainer" containerID="3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33"
Jul 02 20:30:08.538868 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:30:08.538850    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33\": container with ID starting with 3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33 not found: ID does not exist" containerID="3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33"
Jul 02 20:30:08.538917 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.538876    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33"} err="failed to get container status \"3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33\": rpc error: code = NotFound desc = could not find container \"3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33\": container with ID starting with 3050a106881fa89adf0b4e7d0b916e3aef8bc783cbefef899044f83ea79f9d33 not found: ID does not exist"
Jul 02 20:30:08.548402 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.548380    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"]
Jul 02 20:30:08.552157 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:08.552138    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-55ba03-cb9f645b7-4jpkw"]
Jul 02 20:30:10.368805 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:10.368776    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="277128bb-fe88-4a49-9190-44839e69ac6b" path="/var/lib/kubelet/pods/277128bb-fe88-4a49-9190-44839e69ac6b/volumes"
Jul 02 20:30:49.781262 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:49.781178    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"]
Jul 02 20:30:49.781782 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:49.781483    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy" containerID="cri-o://548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf" gracePeriod=30
Jul 02 20:30:51.066826 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.066782    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"]
Jul 02 20:30:51.067527 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.067489    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2" podUID="8c823cb7-d04b-4b58-b824-8a80fe91a8a2" containerName="maas-api" containerID="cri-o://73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba" gracePeriod=30
Jul 02 20:30:51.321333 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.321239    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:30:51.356436 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.356403    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-openshift-service-ca\") pod \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") "
Jul 02 20:30:51.356606 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.356488    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v25rb\" (UniqueName: \"kubernetes.io/projected/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-kube-api-access-v25rb\") pod \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") "
Jul 02 20:30:51.356606 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.356548    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls\") pod \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\" (UID: \"8c823cb7-d04b-4b58-b824-8a80fe91a8a2\") "
Jul 02 20:30:51.356842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.356814    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c823cb7-d04b-4b58-b824-8a80fe91a8a2" (UID: "8c823cb7-d04b-4b58-b824-8a80fe91a8a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:30:51.359025 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.358992    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c823cb7-d04b-4b58-b824-8a80fe91a8a2" (UID: "8c823cb7-d04b-4b58-b824-8a80fe91a8a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:30:51.359025 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.359015    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-kube-api-access-v25rb" (OuterVolumeSpecName: "kube-api-access-v25rb") pod "8c823cb7-d04b-4b58-b824-8a80fe91a8a2" (UID: "8c823cb7-d04b-4b58-b824-8a80fe91a8a2"). InnerVolumeSpecName "kube-api-access-v25rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:30:51.457302 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.457261    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:51.457485 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.457327    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:51.457485 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.457337    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v25rb\" (UniqueName: \"kubernetes.io/projected/8c823cb7-d04b-4b58-b824-8a80fe91a8a2-kube-api-access-v25rb\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:51.735710 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.735672    2536 generic.go:356] "Generic (PLEG): container finished" podID="8c823cb7-d04b-4b58-b824-8a80fe91a8a2" containerID="73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba" exitCode=0
Jul 02 20:30:51.735892 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.735717    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2" event={"ID":"8c823cb7-d04b-4b58-b824-8a80fe91a8a2","Type":"ContainerDied","Data":"73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba"}
Jul 02 20:30:51.735892 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.735761    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2" event={"ID":"8c823cb7-d04b-4b58-b824-8a80fe91a8a2","Type":"ContainerDied","Data":"45605540a1210f62d25fe2dc50ccff873efd13966fbd20e66815735785b43135"}
Jul 02 20:30:51.735892 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.735782    2536 scope.go:117] "RemoveContainer" containerID="73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba"
Jul 02 20:30:51.735892 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.735723    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"
Jul 02 20:30:51.745323 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.745280    2536 scope.go:117] "RemoveContainer" containerID="73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba"
Jul 02 20:30:51.745708 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:30:51.745685    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba\": container with ID starting with 73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba not found: ID does not exist" containerID="73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba"
Jul 02 20:30:51.745786 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.745720    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba"} err="failed to get container status \"73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba\": rpc error: code = NotFound desc = could not find container \"73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba\": container with ID starting with 73a9b3226ac669f0f4e8e2d097c775d0811850c16918aff0133b9de711a983ba not found: ID does not exist"
Jul 02 20:30:51.756867 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.756830    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"]
Jul 02 20:30:51.761218 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:51.761185    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-fe96cd-f7fd8468c-dsdm2"]
Jul 02 20:30:52.368967 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:52.368932    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c823cb7-d04b-4b58-b824-8a80fe91a8a2" path="/var/lib/kubelet/pods/8c823cb7-d04b-4b58-b824-8a80fe91a8a2/volumes"
Jul 02 20:30:52.785892 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:52.785859    2536 patch_prober.go:28] interesting pod/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 02 20:30:52.786059 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:52.785917    2536 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 20:30:55.048092 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.048013    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:30:55.084870 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.084838    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-token\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085077 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.084897    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-credential-socket\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085077 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.084938    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff10733-bd0f-4cd9-a672-952abf18af25-istiod-ca-cert\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085190 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085073    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-certs\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085190 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085151    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-envoy\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085327 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085190    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmpq5\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-kube-api-access-dmpq5\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085327 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085213    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-socket\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085327 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085245    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-data\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085327 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085271    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-podinfo\") pod \"4ff10733-bd0f-4cd9-a672-952abf18af25\" (UID: \"4ff10733-bd0f-4cd9-a672-952abf18af25\") "
Jul 02 20:30:55.085327 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085307    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:30:55.085596 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085353    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:30:55.085596 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085386    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ff10733-bd0f-4cd9-a672-952abf18af25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:30:55.085596 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085570    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:30:55.085768 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085614    2536 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-credential-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.085768 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085631    2536 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff10733-bd0f-4cd9-a672-952abf18af25-istiod-ca-cert\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.085768 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.085645    2536 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.086313 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.086273    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:30:55.087593 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.087566    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:30:55.087976 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.087955    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-kube-api-access-dmpq5" (OuterVolumeSpecName: "kube-api-access-dmpq5") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "kube-api-access-dmpq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:30:55.088144 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.088125    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 20:30:55.088199 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.088152    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ff10733-bd0f-4cd9-a672-952abf18af25" (UID: "4ff10733-bd0f-4cd9-a672-952abf18af25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:30:55.186508 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.186462    2536 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-envoy\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.186508 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.186499    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmpq5\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-kube-api-access-dmpq5\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.186508 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.186513    2536 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-workload-socket\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.186770 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.186525    2536 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-data\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.186770 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.186537    2536 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-podinfo\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.186770 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.186547    2536 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff10733-bd0f-4cd9-a672-952abf18af25-istio-token\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:30:55.756027 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.755988    2536 generic.go:356] "Generic (PLEG): container finished" podID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerID="548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf" exitCode=0
Jul 02 20:30:55.756175 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.756044    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"
Jul 02 20:30:55.756175 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.756061    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" event={"ID":"4ff10733-bd0f-4cd9-a672-952abf18af25","Type":"ContainerDied","Data":"548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf"}
Jul 02 20:30:55.756175 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.756104    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp" event={"ID":"4ff10733-bd0f-4cd9-a672-952abf18af25","Type":"ContainerDied","Data":"e501d0a301124baa5a05e6f654c78e5fb910723d251b2ca970bb8abf8bf62fd4"}
Jul 02 20:30:55.756175 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.756124    2536 scope.go:117] "RemoveContainer" containerID="548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf"
Jul 02 20:30:55.765985 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.765967    2536 scope.go:117] "RemoveContainer" containerID="218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6"
Jul 02 20:30:55.773577 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.773557    2536 scope.go:117] "RemoveContainer" containerID="548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf"
Jul 02 20:30:55.773818 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:30:55.773801    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf\": container with ID starting with 548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf not found: ID does not exist" containerID="548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf"
Jul 02 20:30:55.773864 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.773827    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf"} err="failed to get container status \"548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf\": rpc error: code = NotFound desc = could not find container \"548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf\": container with ID starting with 548c276b8f41cc514dfe8c77c55c713e22b8655cf3cfe4e520598b2bf6af3bdf not found: ID does not exist"
Jul 02 20:30:55.773864 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.773845    2536 scope.go:117] "RemoveContainer" containerID="218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6"
Jul 02 20:30:55.774076 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:30:55.774059    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6\": container with ID starting with 218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6 not found: ID does not exist" containerID="218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6"
Jul 02 20:30:55.774113 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.774083    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6"} err="failed to get container status \"218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6\": rpc error: code = NotFound desc = could not find container \"218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6\": container with ID starting with 218e616a9bc9bc35905ad6da8c76271c73b067366250a81c65fcfbf47dd4d4e6 not found: ID does not exist"
Jul 02 20:30:55.778329 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.778309    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"]
Jul 02 20:30:55.781839 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:55.781819    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-55ba03-openshift-default-784c47c549-wdgpp"]
Jul 02 20:30:56.371077 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:30:56.371042    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" path="/var/lib/kubelet/pods/4ff10733-bd0f-4cd9-a672-952abf18af25/volumes"
Jul 02 20:31:01.152899 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.152865    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"]
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153254    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153266    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153311    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c823cb7-d04b-4b58-b824-8a80fe91a8a2" containerName="maas-api"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153318    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c823cb7-d04b-4b58-b824-8a80fe91a8a2" containerName="maas-api"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153330    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277128bb-fe88-4a49-9190-44839e69ac6b" containerName="maas-api"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153336    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="277128bb-fe88-4a49-9190-44839e69ac6b" containerName="maas-api"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153342    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153346    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153417    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153424    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c823cb7-d04b-4b58-b824-8a80fe91a8a2" containerName="maas-api"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153431    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ff10733-bd0f-4cd9-a672-952abf18af25" containerName="istio-proxy"
Jul 02 20:31:01.153483 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.153439    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="277128bb-fe88-4a49-9190-44839e69ac6b" containerName="maas-api"
Jul 02 20:31:01.158070 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.158042    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.160956 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.160930    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-9f6297-serving-cert\""
Jul 02 20:31:01.169072 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.169043    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"]
Jul 02 20:31:01.235009 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.234966    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glzfr\" (UniqueName: \"kubernetes.io/projected/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-kube-api-access-glzfr\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.235185 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.235040    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.235185 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.235064    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-openshift-service-ca\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.335839 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.335806    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glzfr\" (UniqueName: \"kubernetes.io/projected/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-kube-api-access-glzfr\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.336062 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.335871    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.336062 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:01.335957    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-9f6297-serving-cert: secret "maas-api-e2e-api-a-9f6297-serving-cert" not found
Jul 02 20:31:01.336062 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.335993    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-openshift-service-ca\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.336062 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:01.336016    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls podName:0cdf8bec-a02c-4b25-922f-14cb5c8794ad nodeName:}" failed. No retries permitted until 2026-07-02 20:31:01.835999559 +0000 UTC m=+2073.983768317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls") pod "maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q" (UID: "0cdf8bec-a02c-4b25-922f-14cb5c8794ad") : secret "maas-api-e2e-api-a-9f6297-serving-cert" not found
Jul 02 20:31:01.336663 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.336644    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-openshift-service-ca\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.347181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.347141    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glzfr\" (UniqueName: \"kubernetes.io/projected/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-kube-api-access-glzfr\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.841475 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.841434    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:01.844061 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:01.844029    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls\") pod \"maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") " pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:02.071365 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:02.071321    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:02.219224 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:02.219193    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"]
Jul 02 20:31:02.221571 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:31:02.221542    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cdf8bec_a02c_4b25_922f_14cb5c8794ad.slice/crio-14b0a0f3a51c2450b87df5cdcd59ac7283462e071693b9172f5364c5d9836248 WatchSource:0}: Error finding container 14b0a0f3a51c2450b87df5cdcd59ac7283462e071693b9172f5364c5d9836248: Status 404 returned error can't find the container with id 14b0a0f3a51c2450b87df5cdcd59ac7283462e071693b9172f5364c5d9836248
Jul 02 20:31:02.796577 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:02.796547    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q" event={"ID":"0cdf8bec-a02c-4b25-922f-14cb5c8794ad","Type":"ContainerStarted","Data":"14b0a0f3a51c2450b87df5cdcd59ac7283462e071693b9172f5364c5d9836248"}
Jul 02 20:31:03.802930 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:03.802889    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q" event={"ID":"0cdf8bec-a02c-4b25-922f-14cb5c8794ad","Type":"ContainerStarted","Data":"003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de"}
Jul 02 20:31:03.803340 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:03.802950    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:03.820501 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:03.820447    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q" podStartSLOduration=2.252072669 podStartE2EDuration="2.820428518s" totalImagesPullingTime="568.355849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:31:03.818227235 +0000 UTC m=+2075.965996015" watchObservedRunningTime="2026-07-02 20:31:03.820428518 +0000 UTC m=+2075.968197297"
Jul 02 20:31:09.130559 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.130522    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"]
Jul 02 20:31:09.134245 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.134218    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.136740 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.136705    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-22355e-serving-cert\""
Jul 02 20:31:09.148181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.148139    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"]
Jul 02 20:31:09.206232 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.206182    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4pcj\" (UniqueName: \"kubernetes.io/projected/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-kube-api-access-g4pcj\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.206450 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.206253    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-openshift-service-ca\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.206450 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.206386    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.307315 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.307255    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.307505 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.307356    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4pcj\" (UniqueName: \"kubernetes.io/projected/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-kube-api-access-g4pcj\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.307505 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:09.307402    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-22355e-serving-cert: secret "maas-api-e2e-api-b-22355e-serving-cert" not found
Jul 02 20:31:09.307505 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:09.307484    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls podName:2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6 nodeName:}" failed. No retries permitted until 2026-07-02 20:31:09.8074613 +0000 UTC m=+2081.955230057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls") pod "maas-api-e2e-api-b-22355e-f54b4954f-n4zjl" (UID: "2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6") : secret "maas-api-e2e-api-b-22355e-serving-cert" not found
Jul 02 20:31:09.307644 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.307409    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-openshift-service-ca\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.308062 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.308044    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-openshift-service-ca\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.319035 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.318999    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4pcj\" (UniqueName: \"kubernetes.io/projected/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-kube-api-access-g4pcj\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.813163 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.813123    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:09.813406 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.813368    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:09.815730 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:09.815698    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls\") pod \"maas-api-e2e-api-b-22355e-f54b4954f-n4zjl\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") " pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:10.045831 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:10.045799    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:10.201012 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:10.200929    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"]
Jul 02 20:31:10.203635 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:31:10.203607    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd4447a_a8dd_4ce9_874e_0ef6e1b159b6.slice/crio-e9ac377789de0bfba4dee278207a492f0b2741e7ef98e426bfb9f9b06f6a7f6a WatchSource:0}: Error finding container e9ac377789de0bfba4dee278207a492f0b2741e7ef98e426bfb9f9b06f6a7f6a: Status 404 returned error can't find the container with id e9ac377789de0bfba4dee278207a492f0b2741e7ef98e426bfb9f9b06f6a7f6a
Jul 02 20:31:10.838976 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:10.838927    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl" event={"ID":"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6","Type":"ContainerStarted","Data":"fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e"}
Jul 02 20:31:10.839181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:10.839000    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl" event={"ID":"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6","Type":"ContainerStarted","Data":"e9ac377789de0bfba4dee278207a492f0b2741e7ef98e426bfb9f9b06f6a7f6a"}
Jul 02 20:31:10.839181 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:10.839077    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:10.856408 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:10.856353    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl" podStartSLOduration=1.418855129 podStartE2EDuration="1.856335118s" totalImagesPullingTime="437.479989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:31:10.855257067 +0000 UTC m=+2083.003025848" watchObservedRunningTime="2026-07-02 20:31:10.856335118 +0000 UTC m=+2083.004103950"
Jul 02 20:31:16.849624 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:16.849590    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:22.402479 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.402436    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"]
Jul 02 20:31:22.403139 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.402720    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q" podUID="0cdf8bec-a02c-4b25-922f-14cb5c8794ad" containerName="maas-api" containerID="cri-o://003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de" gracePeriod=30
Jul 02 20:31:22.660839 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.660775    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:22.736191 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.736157    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-openshift-service-ca\") pod \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") "
Jul 02 20:31:22.736422 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.736258    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls\") pod \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") "
Jul 02 20:31:22.736422 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.736343    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glzfr\" (UniqueName: \"kubernetes.io/projected/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-kube-api-access-glzfr\") pod \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\" (UID: \"0cdf8bec-a02c-4b25-922f-14cb5c8794ad\") "
Jul 02 20:31:22.736551 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.736513    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cdf8bec-a02c-4b25-922f-14cb5c8794ad" (UID: "0cdf8bec-a02c-4b25-922f-14cb5c8794ad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:31:22.736734 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.736719    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:31:22.738528 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.738501    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cdf8bec-a02c-4b25-922f-14cb5c8794ad" (UID: "0cdf8bec-a02c-4b25-922f-14cb5c8794ad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:31:22.738634 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.738535    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-kube-api-access-glzfr" (OuterVolumeSpecName: "kube-api-access-glzfr") pod "0cdf8bec-a02c-4b25-922f-14cb5c8794ad" (UID: "0cdf8bec-a02c-4b25-922f-14cb5c8794ad"). InnerVolumeSpecName "kube-api-access-glzfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:31:22.838156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.838120    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glzfr\" (UniqueName: \"kubernetes.io/projected/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-kube-api-access-glzfr\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:31:22.838156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.838150    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdf8bec-a02c-4b25-922f-14cb5c8794ad-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:31:22.898458 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.898416    2536 generic.go:356] "Generic (PLEG): container finished" podID="0cdf8bec-a02c-4b25-922f-14cb5c8794ad" containerID="003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de" exitCode=0
Jul 02 20:31:22.898632 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.898471    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"
Jul 02 20:31:22.898632 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.898472    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q" event={"ID":"0cdf8bec-a02c-4b25-922f-14cb5c8794ad","Type":"ContainerDied","Data":"003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de"}
Jul 02 20:31:22.898632 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.898525    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q" event={"ID":"0cdf8bec-a02c-4b25-922f-14cb5c8794ad","Type":"ContainerDied","Data":"14b0a0f3a51c2450b87df5cdcd59ac7283462e071693b9172f5364c5d9836248"}
Jul 02 20:31:22.898632 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.898547    2536 scope.go:117] "RemoveContainer" containerID="003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de"
Jul 02 20:31:22.908573 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.908481    2536 scope.go:117] "RemoveContainer" containerID="003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de"
Jul 02 20:31:22.908889 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:22.908857    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de\": container with ID starting with 003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de not found: ID does not exist" containerID="003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de"
Jul 02 20:31:22.909024 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.908899    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de"} err="failed to get container status \"003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de\": rpc error: code = NotFound desc = could not find container \"003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de\": container with ID starting with 003cdfbf2266334cbc5065abc3eceda691a0da3550ee34ec7b7afd10c0eb32de not found: ID does not exist"
Jul 02 20:31:22.921917 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.921837    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"]
Jul 02 20:31:22.928169 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:22.928142    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-9f6297-585ccd47c9-6vj7q"]
Jul 02 20:31:24.369568 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:24.369533    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cdf8bec-a02c-4b25-922f-14cb5c8794ad" path="/var/lib/kubelet/pods/0cdf8bec-a02c-4b25-922f-14cb5c8794ad/volumes"
Jul 02 20:31:29.352882 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.352842    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"]
Jul 02 20:31:29.353334 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.353127    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl" podUID="2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" containerName="maas-api" containerID="cri-o://fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e" gracePeriod=30
Jul 02 20:31:29.611777 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.611709    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:29.702650 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.702617    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls\") pod \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") "
Jul 02 20:31:29.702830 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.702737    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4pcj\" (UniqueName: \"kubernetes.io/projected/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-kube-api-access-g4pcj\") pod \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") "
Jul 02 20:31:29.702830 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.702758    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-openshift-service-ca\") pod \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\" (UID: \"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6\") "
Jul 02 20:31:29.703179 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.703150    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" (UID: "2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:31:29.704954 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.704922    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" (UID: "2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:31:29.704954 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.704927    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-kube-api-access-g4pcj" (OuterVolumeSpecName: "kube-api-access-g4pcj") pod "2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" (UID: "2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6"). InnerVolumeSpecName "kube-api-access-g4pcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:31:29.803867 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.803824    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4pcj\" (UniqueName: \"kubernetes.io/projected/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-kube-api-access-g4pcj\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:31:29.803867 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.803862    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:31:29.803867 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.803872    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:31:29.935788 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.935748    2536 generic.go:356] "Generic (PLEG): container finished" podID="2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" containerID="fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e" exitCode=0
Jul 02 20:31:29.935951 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.935795    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl" event={"ID":"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6","Type":"ContainerDied","Data":"fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e"}
Jul 02 20:31:29.935951 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.935807    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"
Jul 02 20:31:29.935951 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.935863    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl" event={"ID":"2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6","Type":"ContainerDied","Data":"e9ac377789de0bfba4dee278207a492f0b2741e7ef98e426bfb9f9b06f6a7f6a"}
Jul 02 20:31:29.935951 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.935884    2536 scope.go:117] "RemoveContainer" containerID="fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e"
Jul 02 20:31:29.945239 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.945219    2536 scope.go:117] "RemoveContainer" containerID="fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e"
Jul 02 20:31:29.945555 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:29.945536    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e\": container with ID starting with fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e not found: ID does not exist" containerID="fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e"
Jul 02 20:31:29.945630 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.945568    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e"} err="failed to get container status \"fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e\": rpc error: code = NotFound desc = could not find container \"fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e\": container with ID starting with fef13a873c0d8e1149cf5dddea33406a67fc97fedead6df2f1da430dfcf34a2e not found: ID does not exist"
Jul 02 20:31:29.956614 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.956587    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"]
Jul 02 20:31:29.960728 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:29.960699    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-22355e-f54b4954f-n4zjl"]
Jul 02 20:31:30.369515 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:30.369432    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" path="/var/lib/kubelet/pods/2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6/volumes"
Jul 02 20:31:39.314870 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.314834    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"]
Jul 02 20:31:39.315459 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.315441    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" containerName="maas-api"
Jul 02 20:31:39.315523 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.315463    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" containerName="maas-api"
Jul 02 20:31:39.315523 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.315505    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cdf8bec-a02c-4b25-922f-14cb5c8794ad" containerName="maas-api"
Jul 02 20:31:39.315523 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.315515    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cdf8bec-a02c-4b25-922f-14cb5c8794ad" containerName="maas-api"
Jul 02 20:31:39.315624 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.315597    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd4447a-a8dd-4ce9-874e-0ef6e1b159b6" containerName="maas-api"
Jul 02 20:31:39.315624 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.315609    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cdf8bec-a02c-4b25-922f-14cb5c8794ad" containerName="maas-api"
Jul 02 20:31:39.318926 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.318904    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.323525 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.323504    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-c4a4b5-serving-cert\""
Jul 02 20:31:39.333201 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.333174    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"]
Jul 02 20:31:39.490440 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.490397    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.490632 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.490463    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c92614b-d47a-4265-90a0-574ca9f1562e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.490632 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.490542    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zjhd\" (UniqueName: \"kubernetes.io/projected/8c92614b-d47a-4265-90a0-574ca9f1562e-kube-api-access-7zjhd\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.592071 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.591969    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zjhd\" (UniqueName: \"kubernetes.io/projected/8c92614b-d47a-4265-90a0-574ca9f1562e-kube-api-access-7zjhd\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.592071 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.592065    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.592336 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.592120    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c92614b-d47a-4265-90a0-574ca9f1562e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.592336 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:39.592231    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-c4a4b5-serving-cert: secret "maas-api-e2e-shared-a-c4a4b5-serving-cert" not found
Jul 02 20:31:39.592336 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:39.592326    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls podName:8c92614b-d47a-4265-90a0-574ca9f1562e nodeName:}" failed. No retries permitted until 2026-07-02 20:31:40.09230891 +0000 UTC m=+2112.240077679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls") pod "maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv" (UID: "8c92614b-d47a-4265-90a0-574ca9f1562e") : secret "maas-api-e2e-shared-a-c4a4b5-serving-cert" not found
Jul 02 20:31:39.592791 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.592772    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c92614b-d47a-4265-90a0-574ca9f1562e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:39.602164 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:39.602143    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zjhd\" (UniqueName: \"kubernetes.io/projected/8c92614b-d47a-4265-90a0-574ca9f1562e-kube-api-access-7zjhd\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:40.097137 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:40.097102    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:40.099814 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:40.099785    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls\") pod \"maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") " pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:40.228828 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:40.228797    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:40.377844 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:40.377822    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"]
Jul 02 20:31:40.379773 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:31:40.379744    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c92614b_d47a_4265_90a0_574ca9f1562e.slice/crio-ce4b3e55e587db43b64894176cbb63db6aa9cd3b0a03736599365554614dc2b6 WatchSource:0}: Error finding container ce4b3e55e587db43b64894176cbb63db6aa9cd3b0a03736599365554614dc2b6: Status 404 returned error can't find the container with id ce4b3e55e587db43b64894176cbb63db6aa9cd3b0a03736599365554614dc2b6
Jul 02 20:31:40.991214 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:40.991124    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv" event={"ID":"8c92614b-d47a-4265-90a0-574ca9f1562e","Type":"ContainerStarted","Data":"a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292"}
Jul 02 20:31:40.991214 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:40.991176    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv" event={"ID":"8c92614b-d47a-4265-90a0-574ca9f1562e","Type":"ContainerStarted","Data":"ce4b3e55e587db43b64894176cbb63db6aa9cd3b0a03736599365554614dc2b6"}
Jul 02 20:31:40.991438 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:40.991220    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:41.007848 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:41.007802    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv" podStartSLOduration=1.683269839 podStartE2EDuration="2.007786849s" totalImagesPullingTime="324.51701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:31:41.007026048 +0000 UTC m=+2113.154794829" watchObservedRunningTime="2026-07-02 20:31:41.007786849 +0000 UTC m=+2113.155555627"
Jul 02 20:31:47.000995 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.000966    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:31:47.383619 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.383581    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"]
Jul 02 20:31:47.387370 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.387348    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.391093 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.391070    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-fd52f9-serving-cert\""
Jul 02 20:31:47.398737 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.398711    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"]
Jul 02 20:31:47.459902 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.459864    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6be7bb4-7cb7-4644-88c6-8967b597dd34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.460070 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.459995    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.460070 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.460026    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldxxc\" (UniqueName: \"kubernetes.io/projected/b6be7bb4-7cb7-4644-88c6-8967b597dd34-kube-api-access-ldxxc\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.560546 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.560512    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6be7bb4-7cb7-4644-88c6-8967b597dd34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.560715 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.560585    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.560715 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.560605    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldxxc\" (UniqueName: \"kubernetes.io/projected/b6be7bb4-7cb7-4644-88c6-8967b597dd34-kube-api-access-ldxxc\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.560792 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:47.560748    2536 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-fd52f9-serving-cert: secret "maas-api-e2e-shared-b-fd52f9-serving-cert" not found
Jul 02 20:31:47.560841 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:31:47.560827    2536 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls podName:b6be7bb4-7cb7-4644-88c6-8967b597dd34 nodeName:}" failed. No retries permitted until 2026-07-02 20:31:48.060807718 +0000 UTC m=+2120.208576475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls") pod "maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62" (UID: "b6be7bb4-7cb7-4644-88c6-8967b597dd34") : secret "maas-api-e2e-shared-b-fd52f9-serving-cert" not found
Jul 02 20:31:47.561156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.561140    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6be7bb4-7cb7-4644-88c6-8967b597dd34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:47.570300 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:47.570270    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldxxc\" (UniqueName: \"kubernetes.io/projected/b6be7bb4-7cb7-4644-88c6-8967b597dd34-kube-api-access-ldxxc\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:48.065195 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:48.065161    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:48.067702 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:48.067682    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls\") pod \"maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") " pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:48.298911 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:48.298876    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:48.437891 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:48.437860    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"]
Jul 02 20:31:48.441641 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:31:48.441615    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6be7bb4_7cb7_4644_88c6_8967b597dd34.slice/crio-abcf8a5e1641ab2f8bad6d3ca5d6312a7785952211da15799bee203fe92fab5c WatchSource:0}: Error finding container abcf8a5e1641ab2f8bad6d3ca5d6312a7785952211da15799bee203fe92fab5c: Status 404 returned error can't find the container with id abcf8a5e1641ab2f8bad6d3ca5d6312a7785952211da15799bee203fe92fab5c
Jul 02 20:31:49.034725 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:49.034636    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62" event={"ID":"b6be7bb4-7cb7-4644-88c6-8967b597dd34","Type":"ContainerStarted","Data":"5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14"}
Jul 02 20:31:49.034725 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:49.034702    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:31:49.034725 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:49.034722    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62" event={"ID":"b6be7bb4-7cb7-4644-88c6-8967b597dd34","Type":"ContainerStarted","Data":"abcf8a5e1641ab2f8bad6d3ca5d6312a7785952211da15799bee203fe92fab5c"}
Jul 02 20:31:49.052007 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:49.051954    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62" podStartSLOduration=1.776383316 podStartE2EDuration="2.051937637s" totalImagesPullingTime="275.554321ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:31:49.050067641 +0000 UTC m=+2121.197836421" watchObservedRunningTime="2026-07-02 20:31:49.051937637 +0000 UTC m=+2121.199706442"
Jul 02 20:31:55.044771 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:31:55.044743    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:32:06.498689 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.498652    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"]
Jul 02 20:32:06.503042 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.503019    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.505628 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.505606    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4a4b5\"/\"kube-root-ca.crt\""
Jul 02 20:32:06.506513 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.506491    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4a4b5\"/\"default-dockercfg-x5n8b\""
Jul 02 20:32:06.506627 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.506491    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4a4b5\"/\"auth-test-model-c4a4b5-kserve-self-signed-certs\""
Jul 02 20:32:06.506627 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.506568    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4a4b5\"/\"openshift-service-ca.crt\""
Jul 02 20:32:06.512109 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.512085    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"]
Jul 02 20:32:06.627963 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.627928    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-home\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.627963 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.627961    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-model-cache\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.628204 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.627989    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-tmp-dir\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.628204 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.628082    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/698de01b-ed2a-4fec-834a-11584fea9dd7-tls-certs\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.628204 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.628142    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rnd9\" (UniqueName: \"kubernetes.io/projected/698de01b-ed2a-4fec-834a-11584fea9dd7-kube-api-access-4rnd9\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.628204 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.628191    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-dshm\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729140 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729100    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/698de01b-ed2a-4fec-834a-11584fea9dd7-tls-certs\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729339 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729168    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rnd9\" (UniqueName: \"kubernetes.io/projected/698de01b-ed2a-4fec-834a-11584fea9dd7-kube-api-access-4rnd9\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729339 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729204    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-dshm\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729339 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729260    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-home\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729339 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729313    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-model-cache\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729503 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729459    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-tmp-dir\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729703 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729680    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-home\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729796 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729776    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-model-cache\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.729859 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.729794    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-tmp-dir\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.731756 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.731736    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-dshm\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.731843 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.731756    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/698de01b-ed2a-4fec-834a-11584fea9dd7-tls-certs\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.738745 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.738727    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rnd9\" (UniqueName: \"kubernetes.io/projected/698de01b-ed2a-4fec-834a-11584fea9dd7-kube-api-access-4rnd9\") pod \"auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") " pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.812943 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.812855    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:06.948793 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:06.948760    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"]
Jul 02 20:32:06.950927 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:32:06.950901    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod698de01b_ed2a_4fec_834a_11584fea9dd7.slice/crio-d9d8b02fe2807f6c8b50dc906b2faa7b7a4a44eab799bd6f78e604acd905d65a WatchSource:0}: Error finding container d9d8b02fe2807f6c8b50dc906b2faa7b7a4a44eab799bd6f78e604acd905d65a: Status 404 returned error can't find the container with id d9d8b02fe2807f6c8b50dc906b2faa7b7a4a44eab799bd6f78e604acd905d65a
Jul 02 20:32:07.118193 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:07.118104    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf" event={"ID":"698de01b-ed2a-4fec-834a-11584fea9dd7","Type":"ContainerStarted","Data":"5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b"}
Jul 02 20:32:07.118193 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:07.118152    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf" event={"ID":"698de01b-ed2a-4fec-834a-11584fea9dd7","Type":"ContainerStarted","Data":"d9d8b02fe2807f6c8b50dc906b2faa7b7a4a44eab799bd6f78e604acd905d65a"}
Jul 02 20:32:07.139527 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:07.139477    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf" podStartSLOduration=1.139462942 podStartE2EDuration="1.139462942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:32:07.138263647 +0000 UTC m=+2139.286032427" watchObservedRunningTime="2026-07-02 20:32:07.139462942 +0000 UTC m=+2139.287231772"
Jul 02 20:32:16.813208 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:16.813167    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:16.813781 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:16.813251    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:16.825517 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:16.825492    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:32:17.184097 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:32:17.184058    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:34:58.144950 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:34:58.144919    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"]
Jul 02 20:35:13.174185 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.174126    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf" podUID="698de01b-ed2a-4fec-834a-11584fea9dd7" containerName="main" containerID="cri-o://5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b" gracePeriod=45
Jul 02 20:35:13.419496 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.419473    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:35:13.469842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.469773    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-model-cache\") pod \"698de01b-ed2a-4fec-834a-11584fea9dd7\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") "
Jul 02 20:35:13.469842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.469826    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-tmp-dir\") pod \"698de01b-ed2a-4fec-834a-11584fea9dd7\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") "
Jul 02 20:35:13.470036 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.469848    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-home\") pod \"698de01b-ed2a-4fec-834a-11584fea9dd7\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") "
Jul 02 20:35:13.470036 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.469866    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/698de01b-ed2a-4fec-834a-11584fea9dd7-tls-certs\") pod \"698de01b-ed2a-4fec-834a-11584fea9dd7\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") "
Jul 02 20:35:13.470036 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.469902    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rnd9\" (UniqueName: \"kubernetes.io/projected/698de01b-ed2a-4fec-834a-11584fea9dd7-kube-api-access-4rnd9\") pod \"698de01b-ed2a-4fec-834a-11584fea9dd7\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") "
Jul 02 20:35:13.470036 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.469920    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-dshm\") pod \"698de01b-ed2a-4fec-834a-11584fea9dd7\" (UID: \"698de01b-ed2a-4fec-834a-11584fea9dd7\") "
Jul 02 20:35:13.470251 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.470092    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-model-cache" (OuterVolumeSpecName: "model-cache") pod "698de01b-ed2a-4fec-834a-11584fea9dd7" (UID: "698de01b-ed2a-4fec-834a-11584fea9dd7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:35:13.470251 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.470113    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-home" (OuterVolumeSpecName: "home") pod "698de01b-ed2a-4fec-834a-11584fea9dd7" (UID: "698de01b-ed2a-4fec-834a-11584fea9dd7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:35:13.470251 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.470199    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-model-cache\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:35:13.470251 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.470215    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-home\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:35:13.470251 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.470233    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "698de01b-ed2a-4fec-834a-11584fea9dd7" (UID: "698de01b-ed2a-4fec-834a-11584fea9dd7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:35:13.472205 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.472184    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-dshm" (OuterVolumeSpecName: "dshm") pod "698de01b-ed2a-4fec-834a-11584fea9dd7" (UID: "698de01b-ed2a-4fec-834a-11584fea9dd7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:35:13.472338 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.472311    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/698de01b-ed2a-4fec-834a-11584fea9dd7-kube-api-access-4rnd9" (OuterVolumeSpecName: "kube-api-access-4rnd9") pod "698de01b-ed2a-4fec-834a-11584fea9dd7" (UID: "698de01b-ed2a-4fec-834a-11584fea9dd7"). InnerVolumeSpecName "kube-api-access-4rnd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:35:13.472387 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.472353    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/698de01b-ed2a-4fec-834a-11584fea9dd7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "698de01b-ed2a-4fec-834a-11584fea9dd7" (UID: "698de01b-ed2a-4fec-834a-11584fea9dd7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:35:13.571127 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.571100    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-tmp-dir\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:35:13.571127 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.571125    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/698de01b-ed2a-4fec-834a-11584fea9dd7-tls-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:35:13.571276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.571137    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rnd9\" (UniqueName: \"kubernetes.io/projected/698de01b-ed2a-4fec-834a-11584fea9dd7-kube-api-access-4rnd9\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:35:13.571276 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:13.571145    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/698de01b-ed2a-4fec-834a-11584fea9dd7-dshm\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:35:14.004114 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.004073    2536 generic.go:356] "Generic (PLEG): container finished" podID="698de01b-ed2a-4fec-834a-11584fea9dd7" containerID="5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b" exitCode=0
Jul 02 20:35:14.004114 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.004120    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"
Jul 02 20:35:14.004425 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.004172    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf" event={"ID":"698de01b-ed2a-4fec-834a-11584fea9dd7","Type":"ContainerDied","Data":"5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b"}
Jul 02 20:35:14.004425 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.004225    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf" event={"ID":"698de01b-ed2a-4fec-834a-11584fea9dd7","Type":"ContainerDied","Data":"d9d8b02fe2807f6c8b50dc906b2faa7b7a4a44eab799bd6f78e604acd905d65a"}
Jul 02 20:35:14.004425 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.004247    2536 scope.go:117] "RemoveContainer" containerID="5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b"
Jul 02 20:35:14.013410 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.013395    2536 scope.go:117] "RemoveContainer" containerID="5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b"
Jul 02 20:35:14.013655 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:35:14.013637    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b\": container with ID starting with 5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b not found: ID does not exist" containerID="5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b"
Jul 02 20:35:14.013718 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.013664    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b"} err="failed to get container status \"5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b\": rpc error: code = NotFound desc = could not find container \"5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b\": container with ID starting with 5c36cab5e8d99dc9595cb8ab327c694dc9471676f8e94616e26f92c5e7c37f3b not found: ID does not exist"
Jul 02 20:35:14.025173 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.025153    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"]
Jul 02 20:35:14.028685 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.028663    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c4a4b5/auth-test-model-c4a4b5-kserve-5696b8c87-7qkrf"]
Jul 02 20:35:14.368712 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:14.368634    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="698de01b-ed2a-4fec-834a-11584fea9dd7" path="/var/lib/kubelet/pods/698de01b-ed2a-4fec-834a-11584fea9dd7/volumes"
Jul 02 20:35:29.805958 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.805919    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"]
Jul 02 20:35:29.806369 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.806307    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="698de01b-ed2a-4fec-834a-11584fea9dd7" containerName="main"
Jul 02 20:35:29.806369 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.806318    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="698de01b-ed2a-4fec-834a-11584fea9dd7" containerName="main"
Jul 02 20:35:29.806449 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.806434    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="698de01b-ed2a-4fec-834a-11584fea9dd7" containerName="main"
Jul 02 20:35:29.811103 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.811083    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:29.814495 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.814474    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"sub-test-model-fd52f9-kserve-self-signed-certs\""
Jul 02 20:35:29.814600 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.814492    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"default-dockercfg-w6cjw\""
Jul 02 20:35:29.814600 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.814525    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"kube-root-ca.crt\""
Jul 02 20:35:29.814600 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.814492    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"openshift-service-ca.crt\""
Jul 02 20:35:29.819640 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.819619    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"]
Jul 02 20:35:29.920694 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.920666    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tls-certs\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:29.920694 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.920696    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-model-cache\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:29.920910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.920718    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tmp-dir\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:29.920910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.920737    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5wlb\" (UniqueName: \"kubernetes.io/projected/7b3e2e71-f6b4-4fd5-8064-35af606c4713-kube-api-access-d5wlb\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:29.920910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.920795    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-dshm\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:29.920910 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:29.920833    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-home\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.021666 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.021626    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-home\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.021836 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.021705    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tls-certs\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.021836 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.021732    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-model-cache\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.021971 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.021858    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tmp-dir\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.021971 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.021892    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5wlb\" (UniqueName: \"kubernetes.io/projected/7b3e2e71-f6b4-4fd5-8064-35af606c4713-kube-api-access-d5wlb\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.021971 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.021959    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-dshm\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.022128 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.022033    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-home\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.022128 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.022082    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-model-cache\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.022223 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.022201    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tmp-dir\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.024099 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.024078    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-dshm\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.024557 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.024535    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tls-certs\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.029667 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.029644    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5wlb\" (UniqueName: \"kubernetes.io/projected/7b3e2e71-f6b4-4fd5-8064-35af606c4713-kube-api-access-d5wlb\") pod \"sub-test-model-fd52f9-kserve-bb8fb979-w9hjg\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") " pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.120986 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.120952    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:30.252215 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:30.252190    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"]
Jul 02 20:35:30.254605 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:35:30.254571    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b3e2e71_f6b4_4fd5_8064_35af606c4713.slice/crio-62dccc419bc0e981b5b64ccc5f1cb9621b09805997e0c47873b5b4c1019c39a9 WatchSource:0}: Error finding container 62dccc419bc0e981b5b64ccc5f1cb9621b09805997e0c47873b5b4c1019c39a9: Status 404 returned error can't find the container with id 62dccc419bc0e981b5b64ccc5f1cb9621b09805997e0c47873b5b4c1019c39a9
Jul 02 20:35:31.087712 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:31.087684    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg" event={"ID":"7b3e2e71-f6b4-4fd5-8064-35af606c4713","Type":"ContainerStarted","Data":"4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85"}
Jul 02 20:35:31.088099 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:31.087720    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg" event={"ID":"7b3e2e71-f6b4-4fd5-8064-35af606c4713","Type":"ContainerStarted","Data":"62dccc419bc0e981b5b64ccc5f1cb9621b09805997e0c47873b5b4c1019c39a9"}
Jul 02 20:35:31.107206 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:31.107161    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg" podStartSLOduration=2.107147091 podStartE2EDuration="2.107147091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:35:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:35:31.10599272 +0000 UTC m=+2343.253761532" watchObservedRunningTime="2026-07-02 20:35:31.107147091 +0000 UTC m=+2343.254915870"
Jul 02 20:35:40.121352 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:40.121317    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:40.121725 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:40.121366    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:40.133502 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:40.133482    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:35:40.145767 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:35:40.145738    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:36:03.397934 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:03.397889    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"]
Jul 02 20:36:18.419037 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.418978    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg" podUID="7b3e2e71-f6b4-4fd5-8064-35af606c4713" containerName="main" containerID="cri-o://4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85" gracePeriod=45
Jul 02 20:36:18.673480 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.673428    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:36:18.769250 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769215    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tmp-dir\") pod \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") "
Jul 02 20:36:18.769250 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769259    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tls-certs\") pod \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") "
Jul 02 20:36:18.769518 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769314    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-model-cache\") pod \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") "
Jul 02 20:36:18.769518 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769367    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5wlb\" (UniqueName: \"kubernetes.io/projected/7b3e2e71-f6b4-4fd5-8064-35af606c4713-kube-api-access-d5wlb\") pod \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") "
Jul 02 20:36:18.769518 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769389    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-home\") pod \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") "
Jul 02 20:36:18.769518 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769417    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-dshm\") pod \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\" (UID: \"7b3e2e71-f6b4-4fd5-8064-35af606c4713\") "
Jul 02 20:36:18.769736 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769550    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b3e2e71-f6b4-4fd5-8064-35af606c4713" (UID: "7b3e2e71-f6b4-4fd5-8064-35af606c4713"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:36:18.769736 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769638    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-home" (OuterVolumeSpecName: "home") pod "7b3e2e71-f6b4-4fd5-8064-35af606c4713" (UID: "7b3e2e71-f6b4-4fd5-8064-35af606c4713"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:36:18.769736 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769653    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b3e2e71-f6b4-4fd5-8064-35af606c4713" (UID: "7b3e2e71-f6b4-4fd5-8064-35af606c4713"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:36:18.769736 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769730    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tmp-dir\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:36:18.769893 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769748    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-model-cache\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:36:18.769893 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.769758    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-home\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:36:18.771664 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.771640    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-dshm" (OuterVolumeSpecName: "dshm") pod "7b3e2e71-f6b4-4fd5-8064-35af606c4713" (UID: "7b3e2e71-f6b4-4fd5-8064-35af606c4713"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:36:18.771664 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.771647    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b3e2e71-f6b4-4fd5-8064-35af606c4713" (UID: "7b3e2e71-f6b4-4fd5-8064-35af606c4713"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:36:18.771811 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.771712    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b3e2e71-f6b4-4fd5-8064-35af606c4713-kube-api-access-d5wlb" (OuterVolumeSpecName: "kube-api-access-d5wlb") pod "7b3e2e71-f6b4-4fd5-8064-35af606c4713" (UID: "7b3e2e71-f6b4-4fd5-8064-35af606c4713"). InnerVolumeSpecName "kube-api-access-d5wlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:36:18.870850 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.870820    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3e2e71-f6b4-4fd5-8064-35af606c4713-tls-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:36:18.870850 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.870846    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5wlb\" (UniqueName: \"kubernetes.io/projected/7b3e2e71-f6b4-4fd5-8064-35af606c4713-kube-api-access-d5wlb\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:36:18.870850 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:18.870855    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3e2e71-f6b4-4fd5-8064-35af606c4713-dshm\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:36:19.321863 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.321829    2536 generic.go:356] "Generic (PLEG): container finished" podID="7b3e2e71-f6b4-4fd5-8064-35af606c4713" containerID="4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85" exitCode=0
Jul 02 20:36:19.322027 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.321874    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"
Jul 02 20:36:19.322027 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.321911    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg" event={"ID":"7b3e2e71-f6b4-4fd5-8064-35af606c4713","Type":"ContainerDied","Data":"4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85"}
Jul 02 20:36:19.322027 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.321957    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg" event={"ID":"7b3e2e71-f6b4-4fd5-8064-35af606c4713","Type":"ContainerDied","Data":"62dccc419bc0e981b5b64ccc5f1cb9621b09805997e0c47873b5b4c1019c39a9"}
Jul 02 20:36:19.322027 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.321973    2536 scope.go:117] "RemoveContainer" containerID="4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85"
Jul 02 20:36:19.331129 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.331112    2536 scope.go:117] "RemoveContainer" containerID="4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85"
Jul 02 20:36:19.331400 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:36:19.331382    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85\": container with ID starting with 4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85 not found: ID does not exist" containerID="4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85"
Jul 02 20:36:19.331460 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.331407    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85"} err="failed to get container status \"4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85\": rpc error: code = NotFound desc = could not find container \"4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85\": container with ID starting with 4a55b44a18d3de699e23f60262987c135b191ac9a3ee2d847e0920f254646a85 not found: ID does not exist"
Jul 02 20:36:19.342475 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.342455    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"]
Jul 02 20:36:19.346344 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:19.346316    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/sub-test-model-fd52f9-kserve-bb8fb979-w9hjg"]
Jul 02 20:36:20.368662 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:20.368631    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b3e2e71-f6b4-4fd5-8064-35af606c4713" path="/var/lib/kubelet/pods/7b3e2e71-f6b4-4fd5-8064-35af606c4713/volumes"
Jul 02 20:36:49.716700 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.716624    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"]
Jul 02 20:36:49.717164 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.717022    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b3e2e71-f6b4-4fd5-8064-35af606c4713" containerName="main"
Jul 02 20:36:49.717164 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.717033    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b3e2e71-f6b4-4fd5-8064-35af606c4713" containerName="main"
Jul 02 20:36:49.717164 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.717109    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b3e2e71-f6b4-4fd5-8064-35af606c4713" containerName="main"
Jul 02 20:36:49.720156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.720140    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.724222 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.724148    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"rate-test-model-fd52f9-kserve-self-signed-certs\""
Jul 02 20:36:49.724222 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.724176    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"openshift-service-ca.crt\""
Jul 02 20:36:49.724418 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.724226    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"kube-root-ca.crt\""
Jul 02 20:36:49.724418 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.724226    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fd52f9\"/\"default-dockercfg-w6cjw\""
Jul 02 20:36:49.728196 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.728173    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"]
Jul 02 20:36:49.820193 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.820165    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tls-certs\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.820380 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.820197    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-model-cache\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.820380 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.820239    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-dshm\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.820380 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.820278    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tmp-dir\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.820380 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.820338    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6s47\" (UniqueName: \"kubernetes.io/projected/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-kube-api-access-g6s47\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.820380 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.820359    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-home\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921309 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921252    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tls-certs\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921309 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921309    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-model-cache\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921348    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-dshm\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921363    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tmp-dir\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921378    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6s47\" (UniqueName: \"kubernetes.io/projected/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-kube-api-access-g6s47\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921555 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921399    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-home\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921810    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-home\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921825    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tmp-dir\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.921969 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.921888    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-model-cache\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.923796 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.923769    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-dshm\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.923894 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.923849    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tls-certs\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:49.929852 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:49.929835    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6s47\" (UniqueName: \"kubernetes.io/projected/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-kube-api-access-g6s47\") pod \"rate-test-model-fd52f9-kserve-576c49d675-bpfmj\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") " pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:50.029854 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:50.029780    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:36:50.163272 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:50.163242    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"]
Jul 02 20:36:50.165113 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:36:50.165082    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f40a13a_9b1d_47e5_aa53_4e9cb401b355.slice/crio-a8a8d44be8eb972ec65fdebad5d2728c5a4ba49a5f71c6551c64afc26906555d WatchSource:0}: Error finding container a8a8d44be8eb972ec65fdebad5d2728c5a4ba49a5f71c6551c64afc26906555d: Status 404 returned error can't find the container with id a8a8d44be8eb972ec65fdebad5d2728c5a4ba49a5f71c6551c64afc26906555d
Jul 02 20:36:50.468642 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:50.468605    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj" event={"ID":"7f40a13a-9b1d-47e5-aa53-4e9cb401b355","Type":"ContainerStarted","Data":"669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90"}
Jul 02 20:36:50.468825 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:50.468653    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj" event={"ID":"7f40a13a-9b1d-47e5-aa53-4e9cb401b355","Type":"ContainerStarted","Data":"a8a8d44be8eb972ec65fdebad5d2728c5a4ba49a5f71c6551c64afc26906555d"}
Jul 02 20:36:50.484849 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:36:50.484805    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj" podStartSLOduration=1.484791391 podStartE2EDuration="1.484791391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 20:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:36:50.484629728 +0000 UTC m=+2422.632398508" watchObservedRunningTime="2026-07-02 20:36:50.484791391 +0000 UTC m=+2422.632560187"
Jul 02 20:37:00.030222 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:00.030187    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:37:00.030727 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:00.030237    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:37:00.042813 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:00.042789    2536 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:37:00.526610 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:00.526585    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:37:34.289587 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:34.289550    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"]
Jul 02 20:37:36.226423 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.226390    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"]
Jul 02 20:37:36.226893 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.226651    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv" podUID="8c92614b-d47a-4265-90a0-574ca9f1562e" containerName="maas-api" containerID="cri-o://a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292" gracePeriod=30
Jul 02 20:37:36.483849 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.483798    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:37:36.533711 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.533686    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c92614b-d47a-4265-90a0-574ca9f1562e-openshift-service-ca\") pod \"8c92614b-d47a-4265-90a0-574ca9f1562e\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") "
Jul 02 20:37:36.533850 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.533717    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls\") pod \"8c92614b-d47a-4265-90a0-574ca9f1562e\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") "
Jul 02 20:37:36.533850 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.533759    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zjhd\" (UniqueName: \"kubernetes.io/projected/8c92614b-d47a-4265-90a0-574ca9f1562e-kube-api-access-7zjhd\") pod \"8c92614b-d47a-4265-90a0-574ca9f1562e\" (UID: \"8c92614b-d47a-4265-90a0-574ca9f1562e\") "
Jul 02 20:37:36.534156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.534114    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c92614b-d47a-4265-90a0-574ca9f1562e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8c92614b-d47a-4265-90a0-574ca9f1562e" (UID: "8c92614b-d47a-4265-90a0-574ca9f1562e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:37:36.535826 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.535800    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c92614b-d47a-4265-90a0-574ca9f1562e-kube-api-access-7zjhd" (OuterVolumeSpecName: "kube-api-access-7zjhd") pod "8c92614b-d47a-4265-90a0-574ca9f1562e" (UID: "8c92614b-d47a-4265-90a0-574ca9f1562e"). InnerVolumeSpecName "kube-api-access-7zjhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:37:36.536210 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.536181    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8c92614b-d47a-4265-90a0-574ca9f1562e" (UID: "8c92614b-d47a-4265-90a0-574ca9f1562e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:37:36.635394 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.635367    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c92614b-d47a-4265-90a0-574ca9f1562e-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:36.635394 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.635390    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c92614b-d47a-4265-90a0-574ca9f1562e-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:36.635556 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.635400    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zjhd\" (UniqueName: \"kubernetes.io/projected/8c92614b-d47a-4265-90a0-574ca9f1562e-kube-api-access-7zjhd\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:36.684858 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.684818    2536 generic.go:356] "Generic (PLEG): container finished" podID="8c92614b-d47a-4265-90a0-574ca9f1562e" containerID="a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292" exitCode=0
Jul 02 20:37:36.685004 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.684864    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"
Jul 02 20:37:36.685004 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.684874    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv" event={"ID":"8c92614b-d47a-4265-90a0-574ca9f1562e","Type":"ContainerDied","Data":"a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292"}
Jul 02 20:37:36.685004 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.684927    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv" event={"ID":"8c92614b-d47a-4265-90a0-574ca9f1562e","Type":"ContainerDied","Data":"ce4b3e55e587db43b64894176cbb63db6aa9cd3b0a03736599365554614dc2b6"}
Jul 02 20:37:36.685004 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.684949    2536 scope.go:117] "RemoveContainer" containerID="a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292"
Jul 02 20:37:36.693582 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.693566    2536 scope.go:117] "RemoveContainer" containerID="a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292"
Jul 02 20:37:36.693879 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:37:36.693854    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292\": container with ID starting with a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292 not found: ID does not exist" containerID="a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292"
Jul 02 20:37:36.693979 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.693882    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292"} err="failed to get container status \"a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292\": rpc error: code = NotFound desc = could not find container \"a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292\": container with ID starting with a612f26d439256dff878a30833fc76fa4ce2bb095b91c0b782e654a6208e0292 not found: ID does not exist"
Jul 02 20:37:36.704915 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.704890    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"]
Jul 02 20:37:36.711175 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:36.711155    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c4a4b5-7d65f5dcf6-lvwrv"]
Jul 02 20:37:38.369275 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:38.369245    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c92614b-d47a-4265-90a0-574ca9f1562e" path="/var/lib/kubelet/pods/8c92614b-d47a-4265-90a0-574ca9f1562e/volumes"
Jul 02 20:37:49.310238 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.310203    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj" podUID="7f40a13a-9b1d-47e5-aa53-4e9cb401b355" containerName="main" containerID="cri-o://669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90" gracePeriod=45
Jul 02 20:37:49.557927 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.557904    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:37:49.649320 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649276    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tls-certs\") pod \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") "
Jul 02 20:37:49.649476 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649341    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6s47\" (UniqueName: \"kubernetes.io/projected/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-kube-api-access-g6s47\") pod \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") "
Jul 02 20:37:49.649476 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649373    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-dshm\") pod \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") "
Jul 02 20:37:49.649476 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649416    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-model-cache\") pod \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") "
Jul 02 20:37:49.649476 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649436    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-home\") pod \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") "
Jul 02 20:37:49.649476 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649461    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tmp-dir\") pod \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\" (UID: \"7f40a13a-9b1d-47e5-aa53-4e9cb401b355\") "
Jul 02 20:37:49.649748 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649636    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-model-cache" (OuterVolumeSpecName: "model-cache") pod "7f40a13a-9b1d-47e5-aa53-4e9cb401b355" (UID: "7f40a13a-9b1d-47e5-aa53-4e9cb401b355"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:37:49.649748 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649664    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-home" (OuterVolumeSpecName: "home") pod "7f40a13a-9b1d-47e5-aa53-4e9cb401b355" (UID: "7f40a13a-9b1d-47e5-aa53-4e9cb401b355"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:37:49.649868 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649761    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7f40a13a-9b1d-47e5-aa53-4e9cb401b355" (UID: "7f40a13a-9b1d-47e5-aa53-4e9cb401b355"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:37:49.649868 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649790    2536 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-model-cache\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:49.649868 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.649807    2536 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-home\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:49.651586 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.651561    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7f40a13a-9b1d-47e5-aa53-4e9cb401b355" (UID: "7f40a13a-9b1d-47e5-aa53-4e9cb401b355"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:37:49.651586 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.651575    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-kube-api-access-g6s47" (OuterVolumeSpecName: "kube-api-access-g6s47") pod "7f40a13a-9b1d-47e5-aa53-4e9cb401b355" (UID: "7f40a13a-9b1d-47e5-aa53-4e9cb401b355"). InnerVolumeSpecName "kube-api-access-g6s47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:37:49.651728 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.651563    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-dshm" (OuterVolumeSpecName: "dshm") pod "7f40a13a-9b1d-47e5-aa53-4e9cb401b355" (UID: "7f40a13a-9b1d-47e5-aa53-4e9cb401b355"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 20:37:49.749126 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.749098    2536 generic.go:356] "Generic (PLEG): container finished" podID="7f40a13a-9b1d-47e5-aa53-4e9cb401b355" containerID="669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90" exitCode=0
Jul 02 20:37:49.749263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.749138    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"
Jul 02 20:37:49.749263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.749185    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj" event={"ID":"7f40a13a-9b1d-47e5-aa53-4e9cb401b355","Type":"ContainerDied","Data":"669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90"}
Jul 02 20:37:49.749263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.749223    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj" event={"ID":"7f40a13a-9b1d-47e5-aa53-4e9cb401b355","Type":"ContainerDied","Data":"a8a8d44be8eb972ec65fdebad5d2728c5a4ba49a5f71c6551c64afc26906555d"}
Jul 02 20:37:49.749263 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.749243    2536 scope.go:117] "RemoveContainer" containerID="669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90"
Jul 02 20:37:49.750176 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.750157    2536 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-dshm\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:49.750277 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.750187    2536 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tmp-dir\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:49.750277 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.750200    2536 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-tls-certs\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:49.750277 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.750211    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6s47\" (UniqueName: \"kubernetes.io/projected/7f40a13a-9b1d-47e5-aa53-4e9cb401b355-kube-api-access-g6s47\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:37:49.758570 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.758554    2536 scope.go:117] "RemoveContainer" containerID="669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90"
Jul 02 20:37:49.758810 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:37:49.758794    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90\": container with ID starting with 669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90 not found: ID does not exist" containerID="669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90"
Jul 02 20:37:49.758853 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.758816    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90"} err="failed to get container status \"669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90\": rpc error: code = NotFound desc = could not find container \"669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90\": container with ID starting with 669d16c92e62225a842992816add14759fe0a3975e61dd2f6a887d06d2886e90 not found: ID does not exist"
Jul 02 20:37:49.770118 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.770094    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"]
Jul 02 20:37:49.773538 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:49.773517    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fd52f9/rate-test-model-fd52f9-kserve-576c49d675-bpfmj"]
Jul 02 20:37:50.368979 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:37:50.368947    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f40a13a-9b1d-47e5-aa53-4e9cb401b355" path="/var/lib/kubelet/pods/7f40a13a-9b1d-47e5-aa53-4e9cb401b355/volumes"
Jul 02 20:38:19.346364 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.346330    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"]
Jul 02 20:38:19.346888 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.346603    2536 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62" podUID="b6be7bb4-7cb7-4644-88c6-8967b597dd34" containerName="maas-api" containerID="cri-o://5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14" gracePeriod=30
Jul 02 20:38:19.592093 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.592066    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:38:19.709228 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.709198    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls\") pod \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") "
Jul 02 20:38:19.709401 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.709343    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldxxc\" (UniqueName: \"kubernetes.io/projected/b6be7bb4-7cb7-4644-88c6-8967b597dd34-kube-api-access-ldxxc\") pod \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") "
Jul 02 20:38:19.709447 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.709406    2536 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6be7bb4-7cb7-4644-88c6-8967b597dd34-openshift-service-ca\") pod \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\" (UID: \"b6be7bb4-7cb7-4644-88c6-8967b597dd34\") "
Jul 02 20:38:19.709752 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.709729    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6be7bb4-7cb7-4644-88c6-8967b597dd34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6be7bb4-7cb7-4644-88c6-8967b597dd34" (UID: "b6be7bb4-7cb7-4644-88c6-8967b597dd34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 20:38:19.711443 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.711421    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6be7bb4-7cb7-4644-88c6-8967b597dd34-kube-api-access-ldxxc" (OuterVolumeSpecName: "kube-api-access-ldxxc") pod "b6be7bb4-7cb7-4644-88c6-8967b597dd34" (UID: "b6be7bb4-7cb7-4644-88c6-8967b597dd34"). InnerVolumeSpecName "kube-api-access-ldxxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 20:38:19.711443 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.711438    2536 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6be7bb4-7cb7-4644-88c6-8967b597dd34" (UID: "b6be7bb4-7cb7-4644-88c6-8967b597dd34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 20:38:19.810381 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.810350    2536 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6be7bb4-7cb7-4644-88c6-8967b597dd34-openshift-service-ca\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:38:19.810381 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.810376    2536 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6be7bb4-7cb7-4644-88c6-8967b597dd34-maas-api-tls\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:38:19.810381 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.810384    2536 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldxxc\" (UniqueName: \"kubernetes.io/projected/b6be7bb4-7cb7-4644-88c6-8967b597dd34-kube-api-access-ldxxc\") on node \"ip-10-0-142-101.ec2.internal\" DevicePath \"\""
Jul 02 20:38:19.907149 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.907115    2536 generic.go:356] "Generic (PLEG): container finished" podID="b6be7bb4-7cb7-4644-88c6-8967b597dd34" containerID="5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14" exitCode=0
Jul 02 20:38:19.907306 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.907160    2536 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"
Jul 02 20:38:19.907306 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.907195    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62" event={"ID":"b6be7bb4-7cb7-4644-88c6-8967b597dd34","Type":"ContainerDied","Data":"5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14"}
Jul 02 20:38:19.907306 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.907249    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62" event={"ID":"b6be7bb4-7cb7-4644-88c6-8967b597dd34","Type":"ContainerDied","Data":"abcf8a5e1641ab2f8bad6d3ca5d6312a7785952211da15799bee203fe92fab5c"}
Jul 02 20:38:19.907306 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.907270    2536 scope.go:117] "RemoveContainer" containerID="5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14"
Jul 02 20:38:19.921623 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.921601    2536 scope.go:117] "RemoveContainer" containerID="5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14"
Jul 02 20:38:19.921881 ip-10-0-142-101 kubenswrapper[2536]: E0702 20:38:19.921863    2536 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14\": container with ID starting with 5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14 not found: ID does not exist" containerID="5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14"
Jul 02 20:38:19.921926 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.921890    2536 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14"} err="failed to get container status \"5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14\": rpc error: code = NotFound desc = could not find container \"5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14\": container with ID starting with 5cb7d1aebc9e4dda6c50a03abde04fc4511ff880fc16318c5b83ddf0f3202a14 not found: ID does not exist"
Jul 02 20:38:19.932170 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.932148    2536 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"]
Jul 02 20:38:19.935924 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:19.935905    2536 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-fd52f9-7d8c887fb8-b4g62"]
Jul 02 20:38:20.369108 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:20.369074    2536 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6be7bb4-7cb7-4644-88c6-8967b597dd34" path="/var/lib/kubelet/pods/b6be7bb4-7cb7-4644-88c6-8967b597dd34/volumes"
Jul 02 20:38:30.984980 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:30.984951    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c5zts_8d39fc83-8f0b-42e4-b01f-19cb107c4a4b/manager/0.log"
Jul 02 20:38:31.107893 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:31.107865    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rszpf_ce4c56eb-3090-42bc-b0ce-a3809301e56b/manager/0.log"
Jul 02 20:38:31.250345 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:31.250258    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6496b86fff-7dhvk_2a36a87b-3ba5-4f97-809f-0c48c20673bd/maas-api/0.log"
Jul 02 20:38:31.663340 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:31.663267    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77558d7458-w89r2_a39235ae-9f85-493e-840a-f04b29f4dc35/manager/0.log"
Jul 02 20:38:31.811987 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:31.811950    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9jffl_ff5d9add-73f8-4038-aaa2-672dadeae79d/server/0.log"
Jul 02 20:38:31.944017 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:31.943941    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hxpt9_8a605196-2ef8-46cb-9d32-5f634f27e4cb/manager/0.log"
Jul 02 20:38:32.060122 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:32.060097    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c8797c58-9w5cd_2bd5f4a8-30c9-44f7-bdcd-09d68df95759/copy-manifests/0.log"
Jul 02 20:38:32.066898 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:32.066879    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c8797c58-9w5cd_2bd5f4a8-30c9-44f7-bdcd-09d68df95759/manager/0.log"
Jul 02 20:38:32.507141 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:32.507117    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-67g7s_5ce510fc-dfb7-49f0-828a-8f72f90bf212/postgres/0.log"
Jul 02 20:38:34.133158 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:34.133126    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dsxjg_ae94fc96-285f-4ec4-9380-76e86b131035/manager/0.log"
Jul 02 20:38:34.272110 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:34.272085    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z59sj_17026fea-7580-4627-8aa5-c9e92d1d3567/manager/0.log"
Jul 02 20:38:34.527221 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:34.527152    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r7w9z_bcc963fb-935d-4f49-9019-f87ded78cbdf/registry-server/0.log"
Jul 02 20:38:34.927643 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:34.927611    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hj8dz_03925802-a366-43e1-9bd7-09031289299a/manager/0.log"
Jul 02 20:38:35.363503 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:35.363430    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zwpwt_b6201a6e-84ee-45a8-9f06-4c04bce06bcb/istio-proxy/0.log"
Jul 02 20:38:35.648757 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:35.648734    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7dc9b6d5-cxmst_28464414-9c2c-493d-9b4f-0c993fa4afa6/kube-auth-proxy/0.log"
Jul 02 20:38:35.927138 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:35.927058    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ph4nv_f0dc4238-60b0-4e2f-bb70-2045b337e151/istio-proxy/1.log"
Jul 02 20:38:36.062810 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:36.062786    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-plmr7_8de0d3e2-25e5-44bf-be59-15196ad11db1/payload-pre-processing/0.log"
Jul 02 20:38:36.223305 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:36.223222    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bjwcc_0dd715f0-aa66-4d09-bf19-b634fa42b163/payload-processing/0.log"
Jul 02 20:38:36.929842 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:36.929816    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8mprf_b969f0ae-f0c9-4666-a804-25ff3429388a/main/0.log"
Jul 02 20:38:37.205347 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:37.205269    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcpk2p_91baca46-756a-417c-8a29-e95cc9de74c5/main/0.log"
Jul 02 20:38:37.470679 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:37.470606    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-b9x8j_3883fa0d-efd2-4805-bb26-957fe4591ea5/main/0.log"
Jul 02 20:38:45.086295 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:45.086249    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-llksn_41653b25-2617-4640-a58f-1853517ff74d/global-pull-secret-syncer/0.log"
Jul 02 20:38:45.212793 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:45.212764    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hvksz_e880c852-03c2-4730-ae0e-26df38f98598/konnectivity-agent/0.log"
Jul 02 20:38:45.343630 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:45.343557    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-101.ec2.internal_6a63f3713d25ef95e553ae4a5cda922f/haproxy/0.log"
Jul 02 20:38:50.599111 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:50.599084    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dsxjg_ae94fc96-285f-4ec4-9380-76e86b131035/manager/0.log"
Jul 02 20:38:50.630158 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:50.630131    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z59sj_17026fea-7580-4627-8aa5-c9e92d1d3567/manager/0.log"
Jul 02 20:38:50.698920 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:50.698890    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r7w9z_bcc963fb-935d-4f49-9019-f87ded78cbdf/registry-server/0.log"
Jul 02 20:38:50.981488 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:50.981452    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hj8dz_03925802-a366-43e1-9bd7-09031289299a/manager/0.log"
Jul 02 20:38:53.127342 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:53.127315    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l254d_e0fef006-37fc-4281-9bb8-d174c74754cd/node-exporter/0.log"
Jul 02 20:38:53.153252 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:53.153228    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l254d_e0fef006-37fc-4281-9bb8-d174c74754cd/kube-rbac-proxy/0.log"
Jul 02 20:38:53.180899 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:53.180872    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l254d_e0fef006-37fc-4281-9bb8-d174c74754cd/init-textfile/0.log"
Jul 02 20:38:55.080372 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.080337    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-9g79g_3e2e2791-aba8-4f69-a931-de18d3887569/networking-console-plugin/0.log"
Jul 02 20:38:55.498003 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.497974    2536 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"]
Jul 02 20:38:55.498375 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498362    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6be7bb4-7cb7-4644-88c6-8967b597dd34" containerName="maas-api"
Jul 02 20:38:55.498375 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498376    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6be7bb4-7cb7-4644-88c6-8967b597dd34" containerName="maas-api"
Jul 02 20:38:55.498471 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498394    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c92614b-d47a-4265-90a0-574ca9f1562e" containerName="maas-api"
Jul 02 20:38:55.498471 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498399    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c92614b-d47a-4265-90a0-574ca9f1562e" containerName="maas-api"
Jul 02 20:38:55.498471 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498406    2536 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f40a13a-9b1d-47e5-aa53-4e9cb401b355" containerName="main"
Jul 02 20:38:55.498471 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498411    2536 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f40a13a-9b1d-47e5-aa53-4e9cb401b355" containerName="main"
Jul 02 20:38:55.498471 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498465    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c92614b-d47a-4265-90a0-574ca9f1562e" containerName="maas-api"
Jul 02 20:38:55.498471 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498473    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6be7bb4-7cb7-4644-88c6-8967b597dd34" containerName="maas-api"
Jul 02 20:38:55.498682 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.498482    2536 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f40a13a-9b1d-47e5-aa53-4e9cb401b355" containerName="main"
Jul 02 20:38:55.501672 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.501653    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.504362 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.504339    2536 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d6xfm\"/\"default-dockercfg-48tzv\""
Jul 02 20:38:55.504362 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.504352    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d6xfm\"/\"kube-root-ca.crt\""
Jul 02 20:38:55.504512 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.504377    2536 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d6xfm\"/\"openshift-service-ca.crt\""
Jul 02 20:38:55.510714 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.510610    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"]
Jul 02 20:38:55.642153 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.642123    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-proc\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.642396 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.642169    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-lib-modules\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.642396 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.642227    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-podres\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.642396 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.642350    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fbqt\" (UniqueName: \"kubernetes.io/projected/60d1e06f-4282-45aa-8c53-c353bea5df63-kube-api-access-4fbqt\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.642396 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.642381    2536 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-sys\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743270 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743239    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-proc\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743456 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743276    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-lib-modules\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743456 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743328    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-podres\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743456 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743368    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-proc\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743456 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743384    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fbqt\" (UniqueName: \"kubernetes.io/projected/60d1e06f-4282-45aa-8c53-c353bea5df63-kube-api-access-4fbqt\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743456 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743415    2536 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-sys\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743635 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743477    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-lib-modules\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743635 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743483    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-podres\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.743635 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.743523    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60d1e06f-4282-45aa-8c53-c353bea5df63-sys\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.751716 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.751654    2536 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fbqt\" (UniqueName: \"kubernetes.io/projected/60d1e06f-4282-45aa-8c53-c353bea5df63-kube-api-access-4fbqt\") pod \"perf-node-gather-daemonset-47gnl\" (UID: \"60d1e06f-4282-45aa-8c53-c353bea5df63\") " pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.812899 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.812870    2536 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:55.943129 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:55.943109    2536 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"]
Jul 02 20:38:55.945485 ip-10-0-142-101 kubenswrapper[2536]: W0702 20:38:55.945456    2536 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod60d1e06f_4282_45aa_8c53_c353bea5df63.slice/crio-91b10c151d8c5b6757fd1e72c4753f3e5b70b05b05544aff9de5a6f4ab470c84 WatchSource:0}: Error finding container 91b10c151d8c5b6757fd1e72c4753f3e5b70b05b05544aff9de5a6f4ab470c84: Status 404 returned error can't find the container with id 91b10c151d8c5b6757fd1e72c4753f3e5b70b05b05544aff9de5a6f4ab470c84
Jul 02 20:38:56.074696 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:56.074666    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl" event={"ID":"60d1e06f-4282-45aa-8c53-c353bea5df63","Type":"ContainerStarted","Data":"29f2fc72d7168211262d4c49e01decfff3a4c1f21c3b15e45f347b83181bde56"}
Jul 02 20:38:56.074833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:56.074722    2536 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl" event={"ID":"60d1e06f-4282-45aa-8c53-c353bea5df63","Type":"ContainerStarted","Data":"91b10c151d8c5b6757fd1e72c4753f3e5b70b05b05544aff9de5a6f4ab470c84"}
Jul 02 20:38:56.074833 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:56.074788    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:38:56.093041 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:56.092997    2536 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl" podStartSLOduration=1.092984645 podStartE2EDuration="1.092984645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 20:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 20:38:56.091470296 +0000 UTC m=+2548.239239076" watchObservedRunningTime="2026-07-02 20:38:56.092984645 +0000 UTC m=+2548.240753441"
Jul 02 20:38:57.926121 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:57.926094    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hwq8h_d625d727-ce77-4086-acd0-b6610616afa8/dns/0.log"
Jul 02 20:38:57.947441 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:57.947419    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hwq8h_d625d727-ce77-4086-acd0-b6610616afa8/kube-rbac-proxy/0.log"
Jul 02 20:38:58.102337 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:58.102313    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bt4zv_b9f3b082-85fb-4e61-84eb-11697e1dbb76/dns-node-resolver/0.log"
Jul 02 20:38:58.650584 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:58.650555    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-89d64b9c5-79mjs_a764676c-2904-4b71-b526-a9d6111b1a67/registry/0.log"
Jul 02 20:38:58.724696 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:58.724664    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xnztd_313b0821-33f6-430d-b1c7-7b1723f5930c/node-ca/0.log"
Jul 02 20:38:59.680168 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:59.680137    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zwpwt_b6201a6e-84ee-45a8-9f06-4c04bce06bcb/istio-proxy/0.log"
Jul 02 20:38:59.902615 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:59.902586    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7dc9b6d5-cxmst_28464414-9c2c-493d-9b4f-0c993fa4afa6/kube-auth-proxy/0.log"
Jul 02 20:38:59.955889 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:59.955800    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ph4nv_f0dc4238-60b0-4e2f-bb70-2045b337e151/istio-proxy/0.log"
Jul 02 20:38:59.980601 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:38:59.980572    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ph4nv_f0dc4238-60b0-4e2f-bb70-2045b337e151/istio-proxy/1.log"
Jul 02 20:39:00.010132 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:00.010103    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-plmr7_8de0d3e2-25e5-44bf-be59-15196ad11db1/payload-pre-processing/0.log"
Jul 02 20:39:00.058518 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:00.058487    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bjwcc_0dd715f0-aa66-4d09-bf19-b634fa42b163/payload-processing/0.log"
Jul 02 20:39:00.825740 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:00.825715    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j7xqq_d428d5e0-4fd2-40e1-8cd6-be097512e90c/serve-healthcheck-canary/0.log"
Jul 02 20:39:01.504194 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:01.504167    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqcmd_9905aed5-7419-45c0-a28a-aedc1a674c29/kube-rbac-proxy/0.log"
Jul 02 20:39:01.526482 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:01.526456    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqcmd_9905aed5-7419-45c0-a28a-aedc1a674c29/exporter/0.log"
Jul 02 20:39:01.548333 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:01.548308    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqcmd_9905aed5-7419-45c0-a28a-aedc1a674c29/extractor/0.log"
Jul 02 20:39:02.089360 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:02.089276    2536 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d6xfm/perf-node-gather-daemonset-47gnl"
Jul 02 20:39:03.629762 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:03.629732    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c5zts_8d39fc83-8f0b-42e4-b01f-19cb107c4a4b/manager/0.log"
Jul 02 20:39:03.750433 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:03.750401    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rszpf_ce4c56eb-3090-42bc-b0ce-a3809301e56b/manager/0.log"
Jul 02 20:39:03.784273 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:03.784252    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6496b86fff-7dhvk_2a36a87b-3ba5-4f97-809f-0c48c20673bd/maas-api/0.log"
Jul 02 20:39:03.927234 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:03.927203    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77558d7458-w89r2_a39235ae-9f85-493e-840a-f04b29f4dc35/manager/0.log"
Jul 02 20:39:03.950444 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:03.950413    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9jffl_ff5d9add-73f8-4038-aaa2-672dadeae79d/server/0.log"
Jul 02 20:39:04.011737 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:04.011710    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hxpt9_8a605196-2ef8-46cb-9d32-5f634f27e4cb/manager/0.log"
Jul 02 20:39:04.040480 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:04.040456    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c8797c58-9w5cd_2bd5f4a8-30c9-44f7-bdcd-09d68df95759/manager/0.log"
Jul 02 20:39:04.062031 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:04.062010    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66c8797c58-9w5cd_2bd5f4a8-30c9-44f7-bdcd-09d68df95759/copy-manifests/0.log"
Jul 02 20:39:04.280682 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:04.280603    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-67g7s_5ce510fc-dfb7-49f0-828a-8f72f90bf212/postgres/0.log"
Jul 02 20:39:06.090941 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:06.090823    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zwpwt_b6201a6e-84ee-45a8-9f06-4c04bce06bcb/istio-proxy/0.log"
Jul 02 20:39:06.295607 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:06.295578    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7dc9b6d5-cxmst_28464414-9c2c-493d-9b4f-0c993fa4afa6/kube-auth-proxy/0.log"
Jul 02 20:39:06.325055 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:06.325026    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ph4nv_f0dc4238-60b0-4e2f-bb70-2045b337e151/istio-proxy/0.log"
Jul 02 20:39:06.353218 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:06.353140    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ph4nv_f0dc4238-60b0-4e2f-bb70-2045b337e151/istio-proxy/1.log"
Jul 02 20:39:06.375876 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:06.375847    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-plmr7_8de0d3e2-25e5-44bf-be59-15196ad11db1/payload-pre-processing/0.log"
Jul 02 20:39:06.409567 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:06.409542    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bjwcc_0dd715f0-aa66-4d09-bf19-b634fa42b163/payload-processing/0.log"
Jul 02 20:39:07.619009 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:07.618934    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zwpwt_b6201a6e-84ee-45a8-9f06-4c04bce06bcb/istio-proxy/0.log"
Jul 02 20:39:07.825157 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:07.825128    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7dc9b6d5-cxmst_28464414-9c2c-493d-9b4f-0c993fa4afa6/kube-auth-proxy/0.log"
Jul 02 20:39:07.860845 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:07.860813    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ph4nv_f0dc4238-60b0-4e2f-bb70-2045b337e151/istio-proxy/0.log"
Jul 02 20:39:07.887223 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:07.887201    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ph4nv_f0dc4238-60b0-4e2f-bb70-2045b337e151/istio-proxy/1.log"
Jul 02 20:39:07.908840 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:07.908807    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-plmr7_8de0d3e2-25e5-44bf-be59-15196ad11db1/payload-pre-processing/0.log"
Jul 02 20:39:07.948373 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:07.948350    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bjwcc_0dd715f0-aa66-4d09-bf19-b634fa42b163/payload-processing/0.log"
Jul 02 20:39:12.339438 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.339407    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-587br_112bace8-f3ae-4a77-aa75-a40214e6a48d/kube-multus-additional-cni-plugins/0.log"
Jul 02 20:39:12.361206 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.361183    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-587br_112bace8-f3ae-4a77-aa75-a40214e6a48d/egress-router-binary-copy/0.log"
Jul 02 20:39:12.383513 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.383492    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-587br_112bace8-f3ae-4a77-aa75-a40214e6a48d/cni-plugins/0.log"
Jul 02 20:39:12.405043 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.405022    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-587br_112bace8-f3ae-4a77-aa75-a40214e6a48d/bond-cni-plugin/0.log"
Jul 02 20:39:12.427662 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.427638    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-587br_112bace8-f3ae-4a77-aa75-a40214e6a48d/routeoverride-cni/0.log"
Jul 02 20:39:12.450103 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.450082    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-587br_112bace8-f3ae-4a77-aa75-a40214e6a48d/whereabouts-cni-bincopy/0.log"
Jul 02 20:39:12.473035 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.473017    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-587br_112bace8-f3ae-4a77-aa75-a40214e6a48d/whereabouts-cni/0.log"
Jul 02 20:39:12.847921 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.847892    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lsq54_b8a4d7e7-5ed2-4876-9851-72ac034f5f2c/kube-multus/0.log"
Jul 02 20:39:12.997156 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:12.997108    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d2pws_f6881e29-762a-4296-8c61-1b1774a6577a/network-metrics-daemon/0.log"
Jul 02 20:39:13.016533 ip-10-0-142-101 kubenswrapper[2536]: I0702 20:39:13.016508    2536 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-d2pws_f6881e29-762a-4296-8c61-1b1774a6577a/kube-rbac-proxy/0.log"