Jul 22 20:56:57.476798 ip-10-0-131-38 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.844013    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848856    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848868    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848871    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848875    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848878    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848880    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848883    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 20:56:57.919292 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848885    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 20:56:57.912091 ip-10-0-131-38 systemd[1]: Started Kubernetes Kubelet.
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848893    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848896    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848899    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848903    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848907    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848911    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848914    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848917    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848920    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848923    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848926    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848928    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848931    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848933    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848936    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848939    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848942    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848944    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 20:56:57.921002 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848947    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848950    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848952    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848955    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848957    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848960    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848962    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848965    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848967    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848970    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848972    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848974    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848977    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848979    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848982    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848985    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848989    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.848997    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849001    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849004    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 20:56:57.922622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849006    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849009    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849011    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849014    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849017    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849019    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849022    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849024    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849027    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849030    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849032    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849035    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849038    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849041    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849043    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849046    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849048    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849051    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849053    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849056    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 20:56:57.950978 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849059    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849061    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849064    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849067    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849069    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849072    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849074    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849077    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849083    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849085    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849088    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849093    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849096    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849098    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849101    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849103    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849106    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849110    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849113    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 20:56:57.953540 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849115    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849118    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849121    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849123    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849126    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849129    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849131    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849134    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849137    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849139    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849142    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849144    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849147    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849149    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849828    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849835    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849838    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849841    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849844    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 20:56:57.967070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849846    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849851    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849854    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849858    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849862    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849865    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849867    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849870    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849872    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849875    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849878    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849880    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849883    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849886    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849889    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849892    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849895    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849898    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849901    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849904    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 20:56:57.969902 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849907    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849910    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849914    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849916    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849919    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849922    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849925    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849927    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849930    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849933    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849935    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849938    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849949    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849951    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849954    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849957    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849960    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849962    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849965    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849968    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 20:56:57.983166 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849970    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849973    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849975    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849978    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849980    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849983    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849986    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849989    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849992    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849995    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.849998    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850001    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850004    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850006    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850009    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850012    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850014    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850017    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850019    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 20:56:58.164894 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850022    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850025    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850028    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850030    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850033    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850035    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850038    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850041    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850043    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850046    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850049    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850051    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850054    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850056    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850059    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850061    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850065    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850067    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850070    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850072    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 20:56:58.165592 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850074    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850077    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850080    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850082    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850085    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850087    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850090    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850092    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850095    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850098    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850100    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850104    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850108    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850111    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850114    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850449    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850454    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850457    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850460    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850464    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 20:56:58.166635 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850466    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850469    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850472    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850475    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850479    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850482    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850485    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850488    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850491    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850494    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850497    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850500    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850502    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850504    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850507    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850510    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850512    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850515    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850518    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 20:56:58.167250 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850520    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850523    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850525    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850528    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850530    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850533    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850535    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850538    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850541    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850543    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850546    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850549    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850552    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850555    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850557    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850560    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850563    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850565    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850568    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 20:56:58.168051 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850570    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850572    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850575    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850578    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850581    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850583    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850586    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850588    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850591    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850593    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850596    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850598    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850601    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850603    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850606    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850608    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850612    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850615    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850618    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850621    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 20:56:58.168601 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850624    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850626    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850629    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850632    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850634    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850637    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850639    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850644    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850647    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850650    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850653    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850656    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850659    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850662    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850668    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850671    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850674    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850676    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850679    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850683    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 20:56:58.169201 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850685    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850688    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850690    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850693    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850696    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850698    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850701    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850703    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850706    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850709    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850712    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850714    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850717    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850719    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850722    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.850724    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850797    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850812    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850817    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850822    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850825    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 20:56:58.169730 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850829    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850832    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850837    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850840    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850843    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850848    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850851    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850854    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850857    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850860    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850863    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850866    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850869    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850873    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850876    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850879    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850882    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850885    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850893    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850896    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850899    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850902    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850905    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850908    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 20:56:58.170343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850911    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850914    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850918    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850922    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850925    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850928    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850931    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850934    2559 flags.go:64] FLAG: --enable-server="true"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850937    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850941    2559 flags.go:64] FLAG: --event-burst="100"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850943    2559 flags.go:64] FLAG: --event-qps="50"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850947    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850950    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850954    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850957    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850960    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850963    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850966    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850969    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850971    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850974    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850977    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850980    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850983    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850986    2559 flags.go:64] FLAG: --feature-gates=""
Jul 22 20:56:58.171004 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850989    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850992    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850995    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.850998    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851001    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851004    2559 flags.go:64] FLAG: --help="false"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851007    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851010    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851012    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851018    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851021    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851025    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851027    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851030    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851033    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851035    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851038    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851041    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851044    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851047    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851052    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851055    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851058    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851060    2559 flags.go:64] FLAG: --lock-file=""
Jul 22 20:56:58.171828 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851063    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851066    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851068    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851074    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851076    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851079    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851082    2559 flags.go:64] FLAG: --logging-format="text"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851085    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851088    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851091    2559 flags.go:64] FLAG: --manifest-url=""
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851093    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851097    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851100    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851104    2559 flags.go:64] FLAG: --max-pods="110"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851107    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851109    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851112    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851116    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851119    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851122    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851124    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851131    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851134    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851137    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 20:56:58.172981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851139    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851142    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851148    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851150    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851155    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851158    2559 flags.go:64] FLAG: --port="10250"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851161    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851164    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00d040e6174346c2b"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851167    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851170    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851190    2559 flags.go:64] FLAG: --register-node="true"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851195    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851198    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851201    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851204    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851207    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851210    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851213    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851216    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851219    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851221    2559 flags.go:64] FLAG: --runonce="false"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851224    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851227    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851230    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851233    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 20:56:58.173782 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851236    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851241    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851244    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851248    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851250    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851253    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851256    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851258    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851261    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851264    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851267    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851274    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851277    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851280    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851284    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851287    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851290    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851293    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851295    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851298    2559 flags.go:64] FLAG: --v="2"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851302    2559 flags.go:64] FLAG: --version="false"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851305    2559 flags.go:64] FLAG: --vmodule=""
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851309    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.851312    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852479    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 20:56:58.362202 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852489    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852493    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852496    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852499    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852502    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852505    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852508    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852512    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852519    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852522    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852525    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852528    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852532    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852535    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852538    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852541    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852543    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852546    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852550    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852553    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 20:56:58.363021 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852556    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852558    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852565    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852568    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852570    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852573    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852575    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852578    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852580    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852583    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852585    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852588    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852590    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852593    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852596    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852598    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852601    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852603    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852606    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 20:56:58.363622 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852608    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852612    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852615    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852617    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852619    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852622    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852625    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852627    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852630    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852632    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852635    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852640    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852642    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852645    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852647    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852650    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852652    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852655    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852658    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852660    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 20:56:58.364164 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852663    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852665    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852668    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852670    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852673    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852675    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852678    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852681    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852683    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852687    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852691    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852693    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852696    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852700    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852702    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852705    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852707    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852710    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852713    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 20:56:58.364739 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852715    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852718    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852720    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852723    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852728    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852731    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852733    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852737    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852741    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852744    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852747    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852750    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852752    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852755    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852759    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852761    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852764    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852767    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852769    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 20:56:58.365299 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.852772    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.853426    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.860593    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.860606    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.861284    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.861293    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861344    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861349    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861354    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861357    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861361    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861364    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861367    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861369    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861372    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 20:56:58.365811 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861375    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861378    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861383    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861386    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861389    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861392    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861395    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861397    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861400    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861403    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861405    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861408    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861410    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861413    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861416    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861418    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861421    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861425    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861428    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 20:56:58.366326 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861431    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861435    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861438    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861440    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861443    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861446    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861451    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861455    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861458    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861461    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861464    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861467    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861469    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861472    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861475    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861479    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861481    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861484    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861486    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861490    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 20:56:58.366824 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861493    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861496    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861499    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861502    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861504    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861507    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861510    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861512    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861515    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861517    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861520    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861522    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861525    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861527    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861530    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861533    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861535    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861538    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861542    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861544    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 20:56:58.367411 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861547    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861550    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861552    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861555    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861558    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861560    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861563    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861566    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861569    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861571    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861574    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861577    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861580    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861582    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861585    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861587    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861590    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861592    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861594    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861597    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 20:56:58.367931 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861599    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861602    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861604    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861607    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861610    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861612    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861615    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861618    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861620    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861623    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861627    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.861633    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861752    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861757    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861760    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 20:56:58.368512 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861763    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861766    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861770    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861773    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861778    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861781    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861784    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861787    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861790    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861792    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861795    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861797    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861800    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861803    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861805    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861808    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861811    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861813    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861816    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 20:56:58.368929 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861818    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861821    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861823    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861825    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861828    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861830    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861833    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861836    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861839    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861842    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861845    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861847    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861850    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861853    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861856    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861858    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861861    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861864    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861867    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861870    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 20:56:58.369485 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861872    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861875    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861878    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861880    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861883    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861885    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861888    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861891    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861893    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861896    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861898    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861901    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861903    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861906    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861908    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861911    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861913    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861917    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861920    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861925    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 20:56:58.370055 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861927    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861932    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861934    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861937    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861950    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861953    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861955    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861958    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861961    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861964    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861967    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861969    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861972    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861976    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861980    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861983    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861986    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861989    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861992    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 20:56:58.370627 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861995    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.861998    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862000    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862003    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862005    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862008    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862010    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862013    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862015    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862018    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862020    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862023    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862026    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862029    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862033    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862035    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862038    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 20:56:58.371120 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:57.862041    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.862045    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.862147    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.864304    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.865670    2559 server.go:1019] "Starting client certificate rotation"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.865765    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.865803    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.867602    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.868628    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.881125    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.884670    2559 log.go:25] "Validated CRI v1 image API"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.885474    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 20:56:58.371671 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.887573    2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bed53cc9-0eba-418c-8a8e-b67395d2b4cf:/dev/nvme0n1p3 c767f400-33d1-424f-8298-ccea99d87ee1:/dev/nvme0n1p4]
Jul 22 20:56:58.372010 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.887588    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 20:56:58.372107 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.892721    2559 manager.go:216] Machine: {Timestamp:2026-07-22 20:56:57.891575879 +0000 UTC m=+0.292377852 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115098 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2782288001969fc21b445eea1ed7bf SystemUUID:ec278228-8001-969f-c21b-445eea1ed7bf BootID:71d41ade-bc8f-45e6-96a4-ab0a75fbbcdd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ab:3f:c1:bf:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ab:3f:c1:bf:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:20:14:96:10:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 20:56:58.372107 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.892813    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 20:56:58.372107 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.892883    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 20:56:58.372107 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.894261    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 20:56:58.372107 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.895605    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 20:56:58.372107 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.895625    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-38.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 20:56:58.372107 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.896312    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.896321    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.896329    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.896949    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.898516    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.901814    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.901926    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.901959    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.901969    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.901979    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.901988    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.901995    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.902322    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.902339    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.904267    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.906582    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.906609    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907550    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907563    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907569    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907574    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907582    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907586    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907594    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907599    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907607    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907612    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907620    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.907625    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.908303    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.908309    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.911256    2559 server.go:1268] "Started kubelet"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.911341    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.911405    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.911363    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.911752    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-38.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:57.911776    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:57.911800    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-38.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.913215    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.913233    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 20:56:58.372779 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.920254    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztp8j"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:57.919825    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-38.ec2.internal.18c4b824301ebb85  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-38.ec2.internal,UID:ip-10-0-131-38.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-38.ec2.internal,},FirstTimestamp:2026-07-22 20:56:57.911212933 +0000 UTC m=+0.312014907,LastTimestamp:2026-07-22 20:56:57.911212933 +0000 UTC m=+0.312014907,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-38.ec2.internal,}"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:57.922376    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.926338    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ztp8j"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.927518    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.927526    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:57.927908    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928127    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928203    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928126    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928244    2559 factory.go:55] Registering systemd factory
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928258    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928310    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928319    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928466    2559 factory.go:153] Registering CRI-O factory
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928480    2559 factory.go:223] Registration of the crio container factory successfully
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928529    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928552    2559 factory.go:103] Registering Raw factory
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928565    2559 manager.go:1196] Started watching for new ooms in manager
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.928958    2559 manager.go:349] Starting recovery of all containers
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.936685    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.937289    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.938232    2559 manager.go:354] Recovery completed
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:57.939312    2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-38.ec2.internal\" not found" node="ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.942632    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.944321    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.944346    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.944356    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientPID"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.944819    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.944824    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.944828    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.944846    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.945525    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.945548    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.945562    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientPID"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.946001    2559 policy_none.go:49] "None policy: Start"
Jul 22 20:56:58.535099 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.946011    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.946021    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.946899    2559 policy_none.go:47] "Start"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.999741    2559 manager.go:337] "Starting Device Plugin manager"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:57.999799    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.999812    2559 server.go:87] "Starting device plugin registration server"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:57.999994    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.000003    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.000128    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.000226    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.000235    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.000565    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.000603    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.070584    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.070623    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.070643    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.070678    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.072808    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.100908    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.101691    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.101716    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.101726    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientPID"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.101744    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.110242    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.110259    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-38.ec2.internal\": node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.124199    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.171555    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal"]
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.171719    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.173079    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.173101    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.173113    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientPID"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.174457    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.174581    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.174605    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.175075    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.175096    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.175111    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientPID"
Jul 22 20:56:58.537798 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.175145    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.175160    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.175173    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientPID"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.176587    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.176616    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.177251    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.177282    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.177296    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeHasSufficientPID"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.206929    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-38.ec2.internal\" not found" node="ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.210951    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-38.ec2.internal\" not found" node="ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.224753    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.229590    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/429c95077d85854869871ef21b735a84-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal\" (UID: \"429c95077d85854869871ef21b735a84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.229616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae44a980b4949927e8289bffa3154c50-config\") pod \"kube-apiserver-proxy-ip-10-0-131-38.ec2.internal\" (UID: \"ae44a980b4949927e8289bffa3154c50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.229632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/429c95077d85854869871ef21b735a84-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal\" (UID: \"429c95077d85854869871ef21b735a84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.325482    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.329923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae44a980b4949927e8289bffa3154c50-config\") pod \"kube-apiserver-proxy-ip-10-0-131-38.ec2.internal\" (UID: \"ae44a980b4949927e8289bffa3154c50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.329956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/429c95077d85854869871ef21b735a84-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal\" (UID: \"429c95077d85854869871ef21b735a84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.329979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/429c95077d85854869871ef21b735a84-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal\" (UID: \"429c95077d85854869871ef21b735a84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.329991    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae44a980b4949927e8289bffa3154c50-config\") pod \"kube-apiserver-proxy-ip-10-0-131-38.ec2.internal\" (UID: \"ae44a980b4949927e8289bffa3154c50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.538961 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.330022    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/429c95077d85854869871ef21b735a84-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal\" (UID: \"429c95077d85854869871ef21b735a84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.539538 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.330028    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/429c95077d85854869871ef21b735a84-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal\" (UID: \"429c95077d85854869871ef21b735a84\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.539538 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.426323    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.539538 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.510813    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.539538 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.514669    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:58.539538 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.527129    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.772352 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.632790    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.772352 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.734895    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.971145 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.835926    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:58.971145 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.866428    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 20:56:58.971145 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.866553    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 20:56:58.971145 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.866584    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 20:56:58.971145 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.930725    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 20:51:58 +0000 UTC" deadline="2028-01-09 02:29:46.664275981 +0000 UTC"
Jul 22 20:56:58.971145 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:58.930749    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12845h32m47.733529754s"
Jul 22 20:56:58.971145 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:58.937076    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:59.115070 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.038195    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:59.115070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:59.081007    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod429c95077d85854869871ef21b735a84.slice/crio-070c4ce015e67293b443262d10f7d66824f3f2de7c4c7db7c0b9a1b327994879 WatchSource:0}: Error finding container 070c4ce015e67293b443262d10f7d66824f3f2de7c4c7db7c0b9a1b327994879: Status 404 returned error can't find the container with id 070c4ce015e67293b443262d10f7d66824f3f2de7c4c7db7c0b9a1b327994879
Jul 22 20:56:59.115070 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:56:59.081578    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae44a980b4949927e8289bffa3154c50.slice/crio-8a171986186c7b613099329fc47a78b8cbda00e4edbb7999aec3098a042ffe90 WatchSource:0}: Error finding container 8a171986186c7b613099329fc47a78b8cbda00e4edbb7999aec3098a042ffe90: Status 404 returned error can't find the container with id 8a171986186c7b613099329fc47a78b8cbda00e4edbb7999aec3098a042ffe90
Jul 22 20:56:59.115070 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.086469    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 20:56:59.310517 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.138299    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:59.310517 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.239371    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:59.310517 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.257502    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 20:56:59.522481 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.339602    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:59.522481 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.440595    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-38.ec2.internal\" not found"
Jul 22 20:56:59.522481 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.482210    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 20:56:59.710154 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.528210    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:59.710154 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.539056    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 20:56:59.710154 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.539869    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal"
Jul 22 20:56:59.710154 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.547580    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.867898    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.905935    2559 apiserver.go:52] "Watching apiserver"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.914933    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.917502    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-xxtv6","openshift-multus/network-metrics-daemon-b62h4","openshift-network-diagnostics/network-check-target-fwwhl","kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal","openshift-image-registry/node-ca-g2qc8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal","openshift-multus/multus-wk5qr","openshift-network-operator/iptables-alerter-hc6cf","openshift-ovn-kubernetes/ovnkube-node-9tbv4","kube-system/global-pull-secret-syncer-8wfk7","kube-system/konnectivity-agent-9rkmp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml","openshift-cluster-node-tuning-operator/tuned-wx8ts","openshift-dns/node-resolver-7cqsd"]
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.919706    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.919791    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.922988    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.925609    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.925666    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.925927    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 20:56:59.926009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.925941    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 20:56:59.927655 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.926058    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2qvv8\""
Jul 22 20:56:59.927655 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.926384    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:56:59.929111 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.928349    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 20:56:59.929111 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.928634    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-67w5g\""
Jul 22 20:56:59.929111 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.928860    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 20:56:59.929111 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.928922    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 20:56:59.929579 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.929254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:56:59.933868 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.931355    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:56:59.933868 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.931805    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 20:56:59.933868 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.931944    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 20:56:59.933868 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.931963    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 20:56:59.933868 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.932082    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wfqsg\""
Jul 22 20:56:59.933868 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.933382    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 20:56:59.933868 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.933606    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 20:56:59.934560 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.933876    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s2kst\""
Jul 22 20:56:59.934560 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.933962    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 20:56:59.937016 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.935959    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:56:59.939016 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.938216    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 20:56:59.939016 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.938862    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hfv76\""
Jul 22 20:56:59.940250 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.939196    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 20:56:59.940250 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.939423    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 20:56:59.940250 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.939613    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 20:56:59.940250 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.939839    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 20:56:59.940932 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.940277    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 20:56:59.940932 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.940467    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wk5qr"
Jul 22 20:56:59.942096 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.941276    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:56:59.942096 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:56:59.941344    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:56:59.942589 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.942081    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:56:59.943759 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.942919    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 20:56:59.943759 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.943178    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 20:56:59.943759 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.943499    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.943765    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-97jx2\""
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.943962    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944066    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0b7daf0-ad29-4a64-916c-959616657b02-tmp-dir\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e752280-465f-4e6a-83d0-52251da4bc17-serviceca\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944341    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-kubelet-config\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-modprobe-d\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944394    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xknr8\""
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysctl-d\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944424    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-host\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944450    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rxkw\" (UniqueName: \"kubernetes.io/projected/2e752280-465f-4e6a-83d0-52251da4bc17-kube-api-access-6rxkw\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944503    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysconfig\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944532    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-sys\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51731d94-4aad-4d68-842f-4c76f13af541-etc-tuned\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.013527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vgmn\" (UniqueName: \"kubernetes.io/projected/a0b7daf0-ad29-4a64-916c-959616657b02-kube-api-access-8vgmn\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysctl-conf\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-systemd\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-run\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944690    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-var-lib-kubelet\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944750    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51731d94-4aad-4d68-842f-4c76f13af541-tmp\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4n64\" (UniqueName: \"kubernetes.io/projected/51731d94-4aad-4d68-842f-4c76f13af541-kube-api-access-r4n64\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944821    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a0b7daf0-ad29-4a64-916c-959616657b02-hosts-file\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e752280-465f-4e6a-83d0-52251da4bc17-host\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944906    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-dbus\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944961    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-kubernetes\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.944984    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-lib-modules\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.945484    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.945569    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.945820    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.947883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.015306 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.948685    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.948897    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.948966    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6vtmr\""
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.950217    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.950296    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.950840    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.950875    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kxl4c\""
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.963589    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nrxxj"
Jul 22 20:57:00.017117 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:56:59.972348    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nrxxj"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.030904    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045192    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vgmn\" (UniqueName: \"kubernetes.io/projected/a0b7daf0-ad29-4a64-916c-959616657b02-kube-api-access-8vgmn\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045245    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-socket-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045274    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c98afdc1-210f-4b81-b746-025145fd6995-host-slash\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-cni-bin\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045319    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovnkube-config\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045344    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-cni-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-socket-dir-parent\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-os-release\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045416    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-registration-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045440    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-run-ovn-kubernetes\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvr4b\" (UniqueName: \"kubernetes.io/projected/4df97b04-40ba-466d-a735-cd1184db9185-kube-api-access-bvr4b\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045488    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovn-node-metrics-cert\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.045617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-system-cni-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.085204 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.051819    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 20:57:00.087726 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.058018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vgmn\" (UniqueName: \"kubernetes.io/projected/a0b7daf0-ad29-4a64-916c-959616657b02-kube-api-access-8vgmn\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.142730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-daemon-config\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.142770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-multus-certs\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.142807    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.142843    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-kubelet-config\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.142914    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-kubelet-config\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.142944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-sys-fs\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.142979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c98afdc1-210f-4b81-b746-025145fd6995-iptables-alerter-script\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.143021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-k8s-cni-cncf-io\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.143063    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.143093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3512e037-e77e-48b8-9b2a-46365e7fa80d-konnectivity-ca\") pod \"konnectivity-agent-9rkmp\" (UID: \"3512e037-e77e-48b8-9b2a-46365e7fa80d\") " pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.149948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-kubelet\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.149988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-run-netns\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150009    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-node-log\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.166141 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150023    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-slash\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150040    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-cnibin\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-kubelet\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-dbus\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cnibin\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-cni-multus\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysctl-d\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-etc-kubernetes\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150294    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-cni-bin\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150336    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysctl-conf\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150358    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-run\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150357    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-dbus\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51731d94-4aad-4d68-842f-4c76f13af541-tmp\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.168915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysctl-d\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a0b7daf0-ad29-4a64-916c-959616657b02-hosts-file\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150428    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-run\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-etc-selinux\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-systemd\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150481    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a0b7daf0-ad29-4a64-916c-959616657b02-hosts-file\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150488    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-etc-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150513    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysctl-conf\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150544    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-kubernetes\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150596    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-lib-modules\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.150618    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-conf-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-kubernetes\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.169848 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.150799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-lib-modules\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.172182 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.153761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51731d94-4aad-4d68-842f-4c76f13af541-tmp\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.178752    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret podName:8c7417f6-3df6-418d-b90c-5e7f49f92fe9 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:00.650675507 +0000 UTC m=+3.051477471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret") pod "global-pull-secret-syncer-8wfk7" (UID: "8c7417f6-3df6-418d-b90c-5e7f49f92fe9") : object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.178791    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p44wt\" (UniqueName: \"kubernetes.io/projected/77de0270-f368-45b6-8b34-b26c6febbb3a-kube-api-access-p44wt\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.178833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-modprobe-d\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.178867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc5qf\" (UniqueName: \"kubernetes.io/projected/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-kube-api-access-gc5qf\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.178898    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-device-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.178926    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrvvz\" (UniqueName: \"kubernetes.io/projected/db290252-412c-45da-aac0-b9f8bb9cfa51-kube-api-access-xrvvz\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.178951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-cni-netd\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.178976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77de0270-f368-45b6-8b34-b26c6febbb3a-cni-binary-copy\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysconfig\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179027    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-sys\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179054    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179078    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-systemd-units\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.182901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-netns\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-hostroot\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-systemd\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-var-lib-kubelet\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4n64\" (UniqueName: \"kubernetes.io/projected/51731d94-4aad-4d68-842f-4c76f13af541-kube-api-access-r4n64\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179281    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-sysconfig\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179313    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e752280-465f-4e6a-83d0-52251da4bc17-host\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179340    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4kwf\" (UniqueName: \"kubernetes.io/projected/c98afdc1-210f-4b81-b746-025145fd6995-kube-api-access-s4kwf\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d28wh\" (UniqueName: \"kubernetes.io/projected/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-kube-api-access-d28wh\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179424    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-os-release\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0b7daf0-ad29-4a64-916c-959616657b02-tmp-dir\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179476    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e752280-465f-4e6a-83d0-52251da4bc17-serviceca\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179502    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-system-cni-dir\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179526    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3512e037-e77e-48b8-9b2a-46365e7fa80d-agent-certs\") pod \"konnectivity-agent-9rkmp\" (UID: \"3512e037-e77e-48b8-9b2a-46365e7fa80d\") " pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.187477 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179554    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-env-overrides\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovnkube-script-lib\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-host\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179659    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-systemd\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179661    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rxkw\" (UniqueName: \"kubernetes.io/projected/2e752280-465f-4e6a-83d0-52251da4bc17-kube-api-access-6rxkw\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-var-lib-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-var-lib-kubelet\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179846    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-ovn\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179872    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-log-socket\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51731d94-4aad-4d68-842f-4c76f13af541-etc-tuned\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.188649 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179895    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-etc-modprobe-d\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.179965    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-sys\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.180071    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e752280-465f-4e6a-83d0-52251da4bc17-host\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.180179    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/51731d94-4aad-4d68-842f-4c76f13af541-host\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.180269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0b7daf0-ad29-4a64-916c-959616657b02-tmp-dir\") pod \"node-resolver-7cqsd\" (UID: \"a0b7daf0-ad29-4a64-916c-959616657b02\") " pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.180752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e752280-465f-4e6a-83d0-52251da4bc17-serviceca\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.183280    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/51731d94-4aad-4d68-842f-4c76f13af541-etc-tuned\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.185483    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.185513    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.185527    2559 projected.go:196] Error preparing data for projected volume kube-api-access-7mrf9 for pod openshift-network-diagnostics/network-check-target-fwwhl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.185584    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9 podName:41af05a1-9431-4de7-8331-b7488704f7de nodeName:}" failed. No retries permitted until 2026-07-22 20:57:00.685567412 +0000 UTC m=+3.086369390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7mrf9" (UniqueName: "kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9") pod "network-check-target-fwwhl" (UID: "41af05a1-9431-4de7-8331-b7488704f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.187104    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4n64\" (UniqueName: \"kubernetes.io/projected/51731d94-4aad-4d68-842f-4c76f13af541-kube-api-access-r4n64\") pod \"tuned-wx8ts\" (UID: \"51731d94-4aad-4d68-842f-4c76f13af541\") " pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.189555 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.188073    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rxkw\" (UniqueName: \"kubernetes.io/projected/2e752280-465f-4e6a-83d0-52251da4bc17-kube-api-access-6rxkw\") pod \"node-ca-g2qc8\" (UID: \"2e752280-465f-4e6a-83d0-52251da4bc17\") " pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.202253    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal" event={"ID":"429c95077d85854869871ef21b735a84","Type":"ContainerStarted","Data":"070c4ce015e67293b443262d10f7d66824f3f2de7c4c7db7c0b9a1b327994879"}
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.202320    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal" event={"ID":"ae44a980b4949927e8289bffa3154c50","Type":"ContainerStarted","Data":"8a171986186c7b613099329fc47a78b8cbda00e4edbb7999aec3098a042ffe90"}
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.209667    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.241279    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wx8ts"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.251021    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7cqsd"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:00.251385    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51731d94_4aad_4d68_842f_4c76f13af541.slice/crio-848f4eec141aec6816b68d7efbf583a1330bcfab21b4efaa6ecce0b700bd7e3a WatchSource:0}: Error finding container 848f4eec141aec6816b68d7efbf583a1330bcfab21b4efaa6ecce0b700bd7e3a: Status 404 returned error can't find the container with id 848f4eec141aec6816b68d7efbf583a1330bcfab21b4efaa6ecce0b700bd7e3a
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.262829    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g2qc8"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.280732    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrvvz\" (UniqueName: \"kubernetes.io/projected/db290252-412c-45da-aac0-b9f8bb9cfa51-kube-api-access-xrvvz\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.280768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-cni-netd\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.280792    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77de0270-f368-45b6-8b34-b26c6febbb3a-cni-binary-copy\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.280891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-cni-netd\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.280928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.280975    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-systemd-units\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-netns\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-hostroot\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.290132 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4kwf\" (UniqueName: \"kubernetes.io/projected/c98afdc1-210f-4b81-b746-025145fd6995-kube-api-access-s4kwf\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d28wh\" (UniqueName: \"kubernetes.io/projected/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-kube-api-access-d28wh\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-netns\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-hostroot\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-os-release\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-systemd-units\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-os-release\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281297    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-system-cni-dir\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281324    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3512e037-e77e-48b8-9b2a-46365e7fa80d-agent-certs\") pod \"konnectivity-agent-9rkmp\" (UID: \"3512e037-e77e-48b8-9b2a-46365e7fa80d\") " pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281347    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281364    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77de0270-f368-45b6-8b34-b26c6febbb3a-cni-binary-copy\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-env-overrides\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281393    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-system-cni-dir\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovnkube-script-lib\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281429    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.292566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281473    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281506    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-var-lib-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281547    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281554    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-var-lib-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281590    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281607    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-ovn\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-log-socket\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281640    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-socket-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c98afdc1-210f-4b81-b746-025145fd6995-host-slash\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-cni-bin\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovnkube-config\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-cni-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281728    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c98afdc1-210f-4b81-b746-025145fd6995-host-slash\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281753    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-cni-bin\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.295602 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281764    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-env-overrides\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281791    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-socket-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281819    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-ovn\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281851    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-cni-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281860    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281892    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-log-socket\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281895    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-socket-dir-parent\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281921    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-os-release\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281941    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-registration-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-run-ovn-kubernetes\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281973    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.281978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvr4b\" (UniqueName: \"kubernetes.io/projected/4df97b04-40ba-466d-a735-cd1184db9185-kube-api-access-bvr4b\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovnkube-script-lib\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282019    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-os-release\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.296540 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282018    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovn-node-metrics-cert\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282082    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-run-ovn-kubernetes\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-registration-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-system-cni-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282136    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-socket-dir-parent\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-daemon-config\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282208    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-multus-certs\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282252    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-multus-certs\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-system-cni-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-sys-fs\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.282394    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282397    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c98afdc1-210f-4b81-b746-025145fd6995-iptables-alerter-script\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282432    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-k8s-cni-cncf-io\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.371657 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.282455    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:00.782437382 +0000 UTC m=+3.183239367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282454    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-sys-fs\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-run-k8s-cni-cncf-io\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282485    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3512e037-e77e-48b8-9b2a-46365e7fa80d-konnectivity-ca\") pod \"konnectivity-agent-9rkmp\" (UID: \"3512e037-e77e-48b8-9b2a-46365e7fa80d\") " pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282549    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-kubelet\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-run-netns\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282580    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-daemon-config\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-node-log\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282621    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-slash\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-cnibin\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282658    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-run-netns\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-slash\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-kubelet\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.372606 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-kubelet\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cnibin\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282714    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-node-log\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-cni-multus\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cnibin\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282767    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-cnibin\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-etc-kubernetes\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-kubelet\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-cni-bin\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282866    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-etc-selinux\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-systemd\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-etc-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282941    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-conf-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.373339 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283006    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p44wt\" (UniqueName: \"kubernetes.io/projected/77de0270-f368-45b6-8b34-b26c6febbb3a-kube-api-access-p44wt\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3512e037-e77e-48b8-9b2a-46365e7fa80d-konnectivity-ca\") pod \"konnectivity-agent-9rkmp\" (UID: \"3512e037-e77e-48b8-9b2a-46365e7fa80d\") " pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc5qf\" (UniqueName: \"kubernetes.io/projected/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-kube-api-access-gc5qf\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-device-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-run-systemd\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283069    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovnkube-config\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.282804    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-cni-multus\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283119    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-host-var-lib-cni-bin\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283132    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-etc-openvswitch\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283152    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-multus-conf-dir\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283304    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-device-dir\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c98afdc1-210f-4b81-b746-025145fd6995-iptables-alerter-script\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283408    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/db290252-412c-45da-aac0-b9f8bb9cfa51-etc-selinux\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.374269 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.283450    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77de0270-f368-45b6-8b34-b26c6febbb3a-etc-kubernetes\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.284059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3512e037-e77e-48b8-9b2a-46365e7fa80d-agent-certs\") pod \"konnectivity-agent-9rkmp\" (UID: \"3512e037-e77e-48b8-9b2a-46365e7fa80d\") " pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.285435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-ovn-node-metrics-cert\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.289400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d28wh\" (UniqueName: \"kubernetes.io/projected/9cfd05ea-f2a7-4e43-8d07-d83d48c5c945-kube-api-access-d28wh\") pod \"ovnkube-node-9tbv4\" (UID: \"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.290588    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.291810    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvr4b\" (UniqueName: \"kubernetes.io/projected/4df97b04-40ba-466d-a735-cd1184db9185-kube-api-access-bvr4b\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.292116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc5qf\" (UniqueName: \"kubernetes.io/projected/4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5-kube-api-access-gc5qf\") pod \"multus-additional-cni-plugins-xxtv6\" (UID: \"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5\") " pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.292657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p44wt\" (UniqueName: \"kubernetes.io/projected/77de0270-f368-45b6-8b34-b26c6febbb3a-kube-api-access-p44wt\") pod \"multus-wk5qr\" (UID: \"77de0270-f368-45b6-8b34-b26c6febbb3a\") " pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.292777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4kwf\" (UniqueName: \"kubernetes.io/projected/c98afdc1-210f-4b81-b746-025145fd6995-kube-api-access-s4kwf\") pod \"iptables-alerter-hc6cf\" (UID: \"c98afdc1-210f-4b81-b746-025145fd6995\") " pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.299470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wk5qr"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.307834    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xxtv6"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.317208    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:00.374919 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.319414    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrvvz\" (UniqueName: \"kubernetes.io/projected/db290252-412c-45da-aac0-b9f8bb9cfa51-kube-api-access-xrvvz\") pod \"aws-ebs-csi-driver-node-rsmml\" (UID: \"db290252-412c-45da-aac0-b9f8bb9cfa51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.480701 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.480662    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml"
Jul 22 20:57:00.574658 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.574593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hc6cf"
Jul 22 20:57:00.685361 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.685332    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:00.685498 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.685441    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:00.685547 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.685496    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret podName:8c7417f6-3df6-418d-b90c-5e7f49f92fe9 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:01.685480443 +0000 UTC m=+4.086282407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret") pod "global-pull-secret-syncer-8wfk7" (UID: "8c7417f6-3df6-418d-b90c-5e7f49f92fe9") : object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:00.786446 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.786417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:00.786596 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.786468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:00.786596 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.786571    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:00.786691 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.786615    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 20:57:00.786691 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.786639    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 20:57:00.786691 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.786653    2559 projected.go:196] Error preparing data for projected volume kube-api-access-7mrf9 for pod openshift-network-diagnostics/network-check-target-fwwhl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:00.786814 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.786620    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:01.786606476 +0000 UTC m=+4.187408439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:00.786814 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:00.786724    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9 podName:41af05a1-9431-4de7-8331-b7488704f7de nodeName:}" failed. No retries permitted until 2026-07-22 20:57:01.786706981 +0000 UTC m=+4.187508951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7mrf9" (UniqueName: "kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9") pod "network-check-target-fwwhl" (UID: "41af05a1-9431-4de7-8331-b7488704f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:00.974870 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.974776    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 20:52:00 +0000 UTC" deadline="2028-01-17 14:13:16.658824393 +0000 UTC"
Jul 22 20:57:00.974870 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:00.974827    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13049h16m15.684003491s"
Jul 22 20:57:01.152471 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:01.152431    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wx8ts" event={"ID":"51731d94-4aad-4d68-842f-4c76f13af541","Type":"ContainerStarted","Data":"848f4eec141aec6816b68d7efbf583a1330bcfab21b4efaa6ecce0b700bd7e3a"}
Jul 22 20:57:01.392535 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:01.392501    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98afdc1_210f_4b81_b746_025145fd6995.slice/crio-62bd2c26ec4963320463c3041117936d4d1c45edc1747dcde2b886627ea6dba1 WatchSource:0}: Error finding container 62bd2c26ec4963320463c3041117936d4d1c45edc1747dcde2b886627ea6dba1: Status 404 returned error can't find the container with id 62bd2c26ec4963320463c3041117936d4d1c45edc1747dcde2b886627ea6dba1
Jul 22 20:57:01.394700 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:01.394659    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a30d582_b3cb_43a0_b8e3_a8ffed74cbb5.slice/crio-b28bb345f503b5c405574b44e5be4d13d7aa13debd6702836db6d9402403bcbf WatchSource:0}: Error finding container b28bb345f503b5c405574b44e5be4d13d7aa13debd6702836db6d9402403bcbf: Status 404 returned error can't find the container with id b28bb345f503b5c405574b44e5be4d13d7aa13debd6702836db6d9402403bcbf
Jul 22 20:57:01.396542 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:01.396484    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e752280_465f_4e6a_83d0_52251da4bc17.slice/crio-2dae3fe23cd94faf2ad05e6ae7a2b075f2de3431da098e04513fd88fd68b2730 WatchSource:0}: Error finding container 2dae3fe23cd94faf2ad05e6ae7a2b075f2de3431da098e04513fd88fd68b2730: Status 404 returned error can't find the container with id 2dae3fe23cd94faf2ad05e6ae7a2b075f2de3431da098e04513fd88fd68b2730
Jul 22 20:57:01.399319 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:01.399291    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b7daf0_ad29_4a64_916c_959616657b02.slice/crio-15e08e8085bb111f58f780c19b93e3918636c3bc2ccef09e5ea61e0f0913a224 WatchSource:0}: Error finding container 15e08e8085bb111f58f780c19b93e3918636c3bc2ccef09e5ea61e0f0913a224: Status 404 returned error can't find the container with id 15e08e8085bb111f58f780c19b93e3918636c3bc2ccef09e5ea61e0f0913a224
Jul 22 20:57:01.401046 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:01.401020    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3512e037_e77e_48b8_9b2a_46365e7fa80d.slice/crio-7ba8db869cc9e6058fb8648b7a20713755b9ce51a0c86e285a6c677f4118e7dc WatchSource:0}: Error finding container 7ba8db869cc9e6058fb8648b7a20713755b9ce51a0c86e285a6c677f4118e7dc: Status 404 returned error can't find the container with id 7ba8db869cc9e6058fb8648b7a20713755b9ce51a0c86e285a6c677f4118e7dc
Jul 22 20:57:01.401631 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:01.401608    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77de0270_f368_45b6_8b34_b26c6febbb3a.slice/crio-1ea5cd7a5af69bd175cb59751a4a1e38f747fb4157d2e3c06cfd5187539bf3b8 WatchSource:0}: Error finding container 1ea5cd7a5af69bd175cb59751a4a1e38f747fb4157d2e3c06cfd5187539bf3b8: Status 404 returned error can't find the container with id 1ea5cd7a5af69bd175cb59751a4a1e38f747fb4157d2e3c06cfd5187539bf3b8
Jul 22 20:57:01.404661 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:01.404381    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfd05ea_f2a7_4e43_8d07_d83d48c5c945.slice/crio-38ec28d86862652e0473028ec448036c2155aec1535fd28d4904007420741da3 WatchSource:0}: Error finding container 38ec28d86862652e0473028ec448036c2155aec1535fd28d4904007420741da3: Status 404 returned error can't find the container with id 38ec28d86862652e0473028ec448036c2155aec1535fd28d4904007420741da3
Jul 22 20:57:01.693812 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:01.693614    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:01.693911 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.693773    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:01.693911 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.693885    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret podName:8c7417f6-3df6-418d-b90c-5e7f49f92fe9 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:03.69386423 +0000 UTC m=+6.094666193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret") pod "global-pull-secret-syncer-8wfk7" (UID: "8c7417f6-3df6-418d-b90c-5e7f49f92fe9") : object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:01.794812 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:01.794674    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:01.794812 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:01.794727    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:01.795032 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.794849    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:01.795032 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.794851    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 20:57:01.795032 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.794873    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 20:57:01.795032 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.794886    2559 projected.go:196] Error preparing data for projected volume kube-api-access-7mrf9 for pod openshift-network-diagnostics/network-check-target-fwwhl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:01.795032 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.794911    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:03.794892442 +0000 UTC m=+6.195694429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:01.795032 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:01.794929    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9 podName:41af05a1-9431-4de7-8331-b7488704f7de nodeName:}" failed. No retries permitted until 2026-07-22 20:57:03.794919526 +0000 UTC m=+6.195721488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7mrf9" (UniqueName: "kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9") pod "network-check-target-fwwhl" (UID: "41af05a1-9431-4de7-8331-b7488704f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:01.975942 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:01.975828    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 20:52:00 +0000 UTC" deadline="2028-02-22 21:37:53.22313781 +0000 UTC"
Jul 22 20:57:01.975942 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:01.975868    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13920h40m51.247274675s"
Jul 22 20:57:02.071927 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.071883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:02.072084 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:02.072018    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:02.072323 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.072299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:02.072418 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.072299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:02.072477 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:02.072419    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:02.072560 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:02.072534    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:02.159338 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.159241    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wk5qr" event={"ID":"77de0270-f368-45b6-8b34-b26c6febbb3a","Type":"ContainerStarted","Data":"1ea5cd7a5af69bd175cb59751a4a1e38f747fb4157d2e3c06cfd5187539bf3b8"}
Jul 22 20:57:02.163047 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.162929    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"38ec28d86862652e0473028ec448036c2155aec1535fd28d4904007420741da3"}
Jul 22 20:57:02.165358 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.165298    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9rkmp" event={"ID":"3512e037-e77e-48b8-9b2a-46365e7fa80d","Type":"ContainerStarted","Data":"7ba8db869cc9e6058fb8648b7a20713755b9ce51a0c86e285a6c677f4118e7dc"}
Jul 22 20:57:02.167388 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.167335    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g2qc8" event={"ID":"2e752280-465f-4e6a-83d0-52251da4bc17","Type":"ContainerStarted","Data":"2dae3fe23cd94faf2ad05e6ae7a2b075f2de3431da098e04513fd88fd68b2730"}
Jul 22 20:57:02.169524 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.169500    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7cqsd" event={"ID":"a0b7daf0-ad29-4a64-916c-959616657b02","Type":"ContainerStarted","Data":"15e08e8085bb111f58f780c19b93e3918636c3bc2ccef09e5ea61e0f0913a224"}
Jul 22 20:57:02.173318 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.173295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerStarted","Data":"b28bb345f503b5c405574b44e5be4d13d7aa13debd6702836db6d9402403bcbf"}
Jul 22 20:57:02.180697 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.180670    2559 generic.go:356] "Generic (PLEG): container finished" podID="429c95077d85854869871ef21b735a84" containerID="6abe560594429ca190cc0254a474b48364bf9e6bb2a13818823c83b8ef882604" exitCode=0
Jul 22 20:57:02.180797 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.180748    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal" event={"ID":"429c95077d85854869871ef21b735a84","Type":"ContainerDied","Data":"6abe560594429ca190cc0254a474b48364bf9e6bb2a13818823c83b8ef882604"}
Jul 22 20:57:02.201430 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.201401    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal" event={"ID":"ae44a980b4949927e8289bffa3154c50","Type":"ContainerStarted","Data":"cc11103214d69d4e7eb3a0700991de7e8c9333e9dfbfb2ed0fab39055fcfeb77"}
Jul 22 20:57:02.212877 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.212853    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hc6cf" event={"ID":"c98afdc1-210f-4b81-b746-025145fd6995","Type":"ContainerStarted","Data":"62bd2c26ec4963320463c3041117936d4d1c45edc1747dcde2b886627ea6dba1"}
Jul 22 20:57:02.215307 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:02.215284    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml" event={"ID":"db290252-412c-45da-aac0-b9f8bb9cfa51","Type":"ContainerStarted","Data":"f3eee4c2d1f5ae852ef85bc1415c13e76d4632f6a98b8aaa170fab507df4511c"}
Jul 22 20:57:03.234467 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:03.234230    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal" event={"ID":"429c95077d85854869871ef21b735a84","Type":"ContainerStarted","Data":"ccc10aa90695046ee001009cf28fd2aabc0585bf86f7c498206cb661fb84553a"}
Jul 22 20:57:03.248767 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:03.248694    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-38.ec2.internal" podStartSLOduration=4.248675538 podStartE2EDuration="4.248675538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:02.231417063 +0000 UTC m=+4.632219054" watchObservedRunningTime="2026-07-22 20:57:03.248675538 +0000 UTC m=+5.649477522"
Jul 22 20:57:03.713440 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:03.712799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:03.713440 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.713000    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:03.713440 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.713060    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret podName:8c7417f6-3df6-418d-b90c-5e7f49f92fe9 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:07.713041802 +0000 UTC m=+10.113843768 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret") pod "global-pull-secret-syncer-8wfk7" (UID: "8c7417f6-3df6-418d-b90c-5e7f49f92fe9") : object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:03.813636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:03.813686    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.813827    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.813883    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:07.813863734 +0000 UTC m=+10.214665697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.814276    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.814297    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.814309    2559 projected.go:196] Error preparing data for projected volume kube-api-access-7mrf9 for pod openshift-network-diagnostics/network-check-target-fwwhl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:03.814383 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:03.814352    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9 podName:41af05a1-9431-4de7-8331-b7488704f7de nodeName:}" failed. No retries permitted until 2026-07-22 20:57:07.814338073 +0000 UTC m=+10.215140037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7mrf9" (UniqueName: "kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9") pod "network-check-target-fwwhl" (UID: "41af05a1-9431-4de7-8331-b7488704f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:04.072365 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:04.072291    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:04.072524 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:04.072420    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:04.072831 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:04.072810    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:04.072926 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:04.072909    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:04.072981 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:04.072973    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:04.073067 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:04.073051    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:06.071508 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:06.071323    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:06.071508 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:06.071438    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:06.071508 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:06.071323    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:06.071979 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:06.071565    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:06.071979 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:06.071735    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:06.071979 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:06.071881    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:07.748920 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:07.748886    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:07.749380 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.749041    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:07.749380 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.749089    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret podName:8c7417f6-3df6-418d-b90c-5e7f49f92fe9 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:15.74907579 +0000 UTC m=+18.149877751 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret") pod "global-pull-secret-syncer-8wfk7" (UID: "8c7417f6-3df6-418d-b90c-5e7f49f92fe9") : object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:07.850140 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:07.850103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:07.850322 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:07.850157    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:07.850322 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.850313    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:07.850422 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.850314    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 20:57:07.850422 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.850344    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 20:57:07.850422 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.850359    2559 projected.go:196] Error preparing data for projected volume kube-api-access-7mrf9 for pod openshift-network-diagnostics/network-check-target-fwwhl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:07.850422 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.850373    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:15.850358857 +0000 UTC m=+18.251160819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:07.850422 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:07.850411    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9 podName:41af05a1-9431-4de7-8331-b7488704f7de nodeName:}" failed. No retries permitted until 2026-07-22 20:57:15.850394674 +0000 UTC m=+18.251196640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7mrf9" (UniqueName: "kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9") pod "network-check-target-fwwhl" (UID: "41af05a1-9431-4de7-8331-b7488704f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:08.072597 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:08.072525    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:08.072750 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:08.072605    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:08.073049 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:08.073026    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:08.073130 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:08.073114    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:08.074292 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:08.074135    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:08.074292 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:08.074246    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:10.071115 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:10.071075    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:10.071574 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:10.071250    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:10.071641 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:10.071620    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:10.071745 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:10.071717    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:10.071797 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:10.071774    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:10.071864 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:10.071844    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:12.073815 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:12.073788    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:12.073815 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:12.073812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:12.074260 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:12.073819    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:12.074260 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:12.073916    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:12.074260 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:12.074038    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:12.074260 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:12.074130    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:14.071921 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:14.071888    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:14.071921 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:14.071922    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:14.072470 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:14.072009    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:14.072470 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:14.072053    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:14.072470 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:14.072142    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:14.072470 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:14.072263    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:15.816272 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:15.816240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:15.816738 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.816413    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:15.816738 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.816484    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret podName:8c7417f6-3df6-418d-b90c-5e7f49f92fe9 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:31.816464937 +0000 UTC m=+34.217266904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret") pod "global-pull-secret-syncer-8wfk7" (UID: "8c7417f6-3df6-418d-b90c-5e7f49f92fe9") : object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:15.917256 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:15.917222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:15.917256 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:15.917260    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:15.917460 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.917346    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:15.917460 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.917394    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:31.917380072 +0000 UTC m=+34.318182032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:15.917460 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.917391    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 20:57:15.917460 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.917421    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 20:57:15.917460 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.917436    2559 projected.go:196] Error preparing data for projected volume kube-api-access-7mrf9 for pod openshift-network-diagnostics/network-check-target-fwwhl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:15.917705 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:15.917505    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9 podName:41af05a1-9431-4de7-8331-b7488704f7de nodeName:}" failed. No retries permitted until 2026-07-22 20:57:31.91748884 +0000 UTC m=+34.318290829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7mrf9" (UniqueName: "kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9") pod "network-check-target-fwwhl" (UID: "41af05a1-9431-4de7-8331-b7488704f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:16.071924 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:16.071843    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:16.072078 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:16.071856    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:16.072078 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:16.071976    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:16.072078 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:16.071971    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:16.072078 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:16.072063    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:16.072286 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:16.072151    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:18.072464 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:18.072430    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:18.072902 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:18.072531    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:18.072902 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:18.072620    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:18.072902 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:18.072742    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:18.072902 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:18.072786    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:18.072902 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:18.072854    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:20.071505 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:20.071479    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:20.071505 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:20.071505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:20.071968 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:20.071483    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:20.071968 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:20.071577    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:20.071968 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:20.071686    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:20.072256 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:20.072011    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:21.275446 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.274362    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wx8ts" event={"ID":"51731d94-4aad-4d68-842f-4c76f13af541","Type":"ContainerStarted","Data":"547828993c2fc0997767ef1e336395c24feb38c542aa507772fecd35dea885d2"}
Jul 22 20:57:21.278011 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.277987    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml" event={"ID":"db290252-412c-45da-aac0-b9f8bb9cfa51","Type":"ContainerStarted","Data":"05507cde0f9cf69cdf2a86dcde86b89817127e88fdbe27a879915a00a79a0943"}
Jul 22 20:57:21.279374 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.279356    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wk5qr" event={"ID":"77de0270-f368-45b6-8b34-b26c6febbb3a","Type":"ContainerStarted","Data":"e05e4370582d0a512304ae1b2514c1a26e23188739576793fc14f2dbc7ca1114"}
Jul 22 20:57:21.281915 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.281899    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 20:57:21.282215 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.282196    2559 generic.go:356] "Generic (PLEG): container finished" podID="9cfd05ea-f2a7-4e43-8d07-d83d48c5c945" containerID="dbe2741e0226f18e2f0e7c1b46233b6c6a0a0daea9b7959a2754a97c4cb17706" exitCode=1
Jul 22 20:57:21.282299 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.282259    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"525be5abdbc49a40a9449f3bc66c4a1666fa498cb92f7b6687f3ab585a802490"}
Jul 22 20:57:21.282357 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.282319    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"d29e4a6505299eb3f7be34e57e92cb49cb2bc2f1389ff6df27d649847132e9a4"}
Jul 22 20:57:21.282392 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.282358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"17a87875bf8db9c024dc6865ae666f447e6b6e261c575d8e40ea5803e2e26841"}
Jul 22 20:57:21.282435 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.282414    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"e90e0ba356841947fcc66048013579390e1a0ffdaf343fe3d836316363dcd545"}
Jul 22 20:57:21.282485 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.282455    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerDied","Data":"dbe2741e0226f18e2f0e7c1b46233b6c6a0a0daea9b7959a2754a97c4cb17706"}
Jul 22 20:57:21.282537 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.282491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"a8d08953fec59bb16c9e6d83c6ad5fbf149106307e17b4c103638b00cfb1c576"}
Jul 22 20:57:21.283513 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.283492    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9rkmp" event={"ID":"3512e037-e77e-48b8-9b2a-46365e7fa80d","Type":"ContainerStarted","Data":"a045e9a9fe328b9adc331249c2b61c3ef252448f5d38aa71b6cfa45c4112e2e6"}
Jul 22 20:57:21.284757 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.284738    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g2qc8" event={"ID":"2e752280-465f-4e6a-83d0-52251da4bc17","Type":"ContainerStarted","Data":"8e69746f3b312019586086fb33481baf38a860e9ff1bf94b35009eacb920b13c"}
Jul 22 20:57:21.285987 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.285963    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7cqsd" event={"ID":"a0b7daf0-ad29-4a64-916c-959616657b02","Type":"ContainerStarted","Data":"845aad75b190341d96b7cadf16727161ecba9489aa1586eaeea918a54be3256f"}
Jul 22 20:57:21.287341 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.287322    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5" containerID="e79fd9b4a8cafaa3a23d86f74580b94b346d4003550906e95140c15ec29cb4f7" exitCode=0
Jul 22 20:57:21.287425 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.287344    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerDied","Data":"e79fd9b4a8cafaa3a23d86f74580b94b346d4003550906e95140c15ec29cb4f7"}
Jul 22 20:57:21.290491 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.290458    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-38.ec2.internal" podStartSLOduration=22.290448333 podStartE2EDuration="22.290448333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:03.248945111 +0000 UTC m=+5.649747095" watchObservedRunningTime="2026-07-22 20:57:21.290448333 +0000 UTC m=+23.691250315"
Jul 22 20:57:21.290826 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.290798    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wx8ts" podStartSLOduration=3.361902083 podStartE2EDuration="23.290794258s" totalImagesPullingTime="19.928892175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:21.290439867 +0000 UTC m=+23.691241851" watchObservedRunningTime="2026-07-22 20:57:21.290794258 +0000 UTC m=+23.691596263"
Jul 22 20:57:21.321636 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.321599    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9rkmp" podStartSLOduration=4.634623372 podStartE2EDuration="23.321589165s" totalImagesPullingTime="18.686965793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:21.321223794 +0000 UTC m=+23.722025777" watchObservedRunningTime="2026-07-22 20:57:21.321589165 +0000 UTC m=+23.722391148"
Jul 22 20:57:21.333218 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.333190    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7cqsd" podStartSLOduration=4.644853647 podStartE2EDuration="23.333163761s" totalImagesPullingTime="18.688310114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:21.333124307 +0000 UTC m=+23.733926290" watchObservedRunningTime="2026-07-22 20:57:21.333163761 +0000 UTC m=+23.733965744"
Jul 22 20:57:21.345332 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.345302    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g2qc8" podStartSLOduration=4.653888234 podStartE2EDuration="23.345291302s" totalImagesPullingTime="18.691403068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:21.344868338 +0000 UTC m=+23.745670321" watchObservedRunningTime="2026-07-22 20:57:21.345291302 +0000 UTC m=+23.746093285"
Jul 22 20:57:21.362543 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.362514    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wk5qr" podStartSLOduration=4.58544367 podStartE2EDuration="23.362504468s" totalImagesPullingTime="18.777060798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:21.36240761 +0000 UTC m=+23.763209592" watchObservedRunningTime="2026-07-22 20:57:21.362504468 +0000 UTC m=+23.763306451"
Jul 22 20:57:21.508271 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.508225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:21.595277 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:21.595242    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:22.028007 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.027975    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:22.071108 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.071081    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:22.071108 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.071097    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:22.071324 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.071085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:22.071324 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:22.071226    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:22.071324 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:22.071268    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:22.071483 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:22.071345    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:22.287693 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.287615    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 20:57:22.290366 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.290336    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hc6cf" event={"ID":"c98afdc1-210f-4b81-b746-025145fd6995","Type":"ContainerStarted","Data":"61973a90152f6c91ef66105cefc783701bc01bb9c72cbd764f356b69a20d87e3"}
Jul 22 20:57:22.292895 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.292868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml" event={"ID":"db290252-412c-45da-aac0-b9f8bb9cfa51","Type":"ContainerStarted","Data":"dfc884b8361fb8c0e9f2f6490311cd47051f236e07e6b00ba1776926ea022c15"}
Jul 22 20:57:22.307445 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.307400    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hc6cf" podStartSLOduration=5.605934611 podStartE2EDuration="24.307385599s" totalImagesPullingTime="18.701450988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:22.307087269 +0000 UTC m=+24.707889252" watchObservedRunningTime="2026-07-22 20:57:22.307385599 +0000 UTC m=+24.708187581"
Jul 22 20:57:22.424485 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.424454    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:22.526670 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.526642    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:22.725998 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.725927    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:22.781932 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.781903    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:22.841977 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.841944    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:22.942832 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:22.942791    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:23.015677 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:23.015528    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T20:57:22.287633892Z","UUID":"be90c50f-15b2-447b-a5e8-a38fa2ae4efd","Handler":null,"Name":"","Endpoint":""}
Jul 22 20:57:23.017254 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:23.017233    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 20:57:23.017376 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:23.017270    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 20:57:23.298787 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:23.298764    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 20:57:23.299501 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:23.299476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"1eb171d46168a1448bb096f63190bfeb77e25e20f462baf26793c91bbcb61df5"}
Jul 22 20:57:24.071325 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.071090    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:24.071604 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.071095    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:24.071604 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:24.071429    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:24.071604 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.071160    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:24.071604 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:24.071510    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:24.071604 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:24.071580    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:24.286271 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.286237    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:24.304087 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.304052    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml" event={"ID":"db290252-412c-45da-aac0-b9f8bb9cfa51","Type":"ContainerStarted","Data":"dc9e2a9d19301627eb3f02f77a9e6be8423ce1b01d51c5df6fdfdbbe8c61d9c6"}
Jul 22 20:57:24.322308 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.322194    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rsmml" podStartSLOduration=4.601546166 podStartE2EDuration="26.322159009s" totalImagesPullingTime="21.720612843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:24.322021428 +0000 UTC m=+26.722823442" watchObservedRunningTime="2026-07-22 20:57:24.322159009 +0000 UTC m=+26.722960993"
Jul 22 20:57:24.387359 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.387305    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:24.751296 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.751225    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:24.752008 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.751972    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:24.835557 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.835513    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:24.937873 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:24.937844    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:25.238761 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:25.238683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:25.325534 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:25.325509    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:26.071058 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.070890    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:26.071260 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.070939    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:26.071260 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:26.071130    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:26.071260 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.070965    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:26.071260 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:26.071204    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:26.071457 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:26.071280    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:26.092633 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.092606    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:26.191909 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.191887    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:26.311379 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.311330    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 20:57:26.311725 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.311691    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"e94138543a30c702674f1346c46a7069f3eb83301f51ef59130962e9bdf2f8f3"}
Jul 22 20:57:26.311965 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.311931    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:26.312079 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.312010    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:26.312156 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.312137    2559 scope.go:117] "RemoveContainer" containerID="dbe2741e0226f18e2f0e7c1b46233b6c6a0a0daea9b7959a2754a97c4cb17706"
Jul 22 20:57:26.313546 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.313522    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5" containerID="53c944ad159eb00868e3795cbfff75dbf536fe6ef2f659161ae2f1604c0473e1" exitCode=0
Jul 22 20:57:26.313630 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.313561    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerDied","Data":"53c944ad159eb00868e3795cbfff75dbf536fe6ef2f659161ae2f1604c0473e1"}
Jul 22 20:57:26.329043 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:26.329027    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:27.319509 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.319484    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 20:57:27.320106 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.320076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" event={"ID":"9cfd05ea-f2a7-4e43-8d07-d83d48c5c945","Type":"ContainerStarted","Data":"30a6da7d197dd08b19fcbbdd39ff6d420a414b40b2aedb5cecd7b5d65468e774"}
Jul 22 20:57:27.321202 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.321149    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:27.337065 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.337046    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:27.348765 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.348745    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fwwhl"]
Jul 22 20:57:27.348870 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.348835    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:27.348919 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:27.348907    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:27.352138 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.352115    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8wfk7"]
Jul 22 20:57:27.352239 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.352219    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:27.352311 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:27.352294    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:27.354098 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.354058    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4" podStartSLOduration=10.213852428 podStartE2EDuration="29.354041139s" totalImagesPullingTime="19.140188711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:27.352439009 +0000 UTC m=+29.753241024" watchObservedRunningTime="2026-07-22 20:57:27.354041139 +0000 UTC m=+29.754843125"
Jul 22 20:57:27.354832 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.354816    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b62h4"]
Jul 22 20:57:27.354907 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:27.354897    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:27.354988 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:27.354972    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:28.324289 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:28.324117    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5" containerID="0b38ab19e548a3766e27de023ecefe0d09804890d1a589bd0e438b800bf73011" exitCode=0
Jul 22 20:57:28.324482 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:28.324206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerDied","Data":"0b38ab19e548a3766e27de023ecefe0d09804890d1a589bd0e438b800bf73011"}
Jul 22 20:57:29.071106 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:29.071074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:29.071613 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:29.071074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:29.071613 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:29.071172    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:29.071613 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:29.071294    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:29.071613 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:29.071074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:29.071613 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:29.071403    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:29.757898 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:29.757831    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:29.758030 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:29.757949    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 22 20:57:29.758484 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:29.758462    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9rkmp"
Jul 22 20:57:29.902407 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:29.902364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:30.333502 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:30.333474    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5" containerID="ca9b466f2b63df99d4cd9eb71f76b23784e919b6b60f33c794f0ebf7874966bf" exitCode=0
Jul 22 20:57:30.334061 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:30.333557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerDied","Data":"ca9b466f2b63df99d4cd9eb71f76b23784e919b6b60f33c794f0ebf7874966bf"}
Jul 22 20:57:30.491562 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:30.491536    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:30.568492 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:30.568472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:31.071277 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:31.071245    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:31.071453 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:31.071247    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:31.071453 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.071364    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:57:31.071569 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.071464    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8wfk7" podUID="8c7417f6-3df6-418d-b90c-5e7f49f92fe9"
Jul 22 20:57:31.071569 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:31.071248    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:31.071656 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.071582    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fwwhl" podUID="41af05a1-9431-4de7-8331-b7488704f7de"
Jul 22 20:57:31.837492 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:31.837414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:31.837876 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.837571    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:31.837876 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.837644    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret podName:8c7417f6-3df6-418d-b90c-5e7f49f92fe9 nodeName:}" failed. No retries permitted until 2026-07-22 20:58:03.837623606 +0000 UTC m=+66.238425569 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret") pod "global-pull-secret-syncer-8wfk7" (UID: "8c7417f6-3df6-418d-b90c-5e7f49f92fe9") : object "kube-system"/"original-pull-secret" not registered
Jul 22 20:57:31.937739 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:31.937714    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:31.937889 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:31.937755    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:31.937889 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.937858    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:31.937889 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.937881    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 20:57:31.938033 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.937901    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 20:57:31.938033 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.937906    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:58:03.937892201 +0000 UTC m=+66.338694162 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 20:57:31.938033 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.937914    2559 projected.go:196] Error preparing data for projected volume kube-api-access-7mrf9 for pod openshift-network-diagnostics/network-check-target-fwwhl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:31.938033 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:31.937964    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9 podName:41af05a1-9431-4de7-8331-b7488704f7de nodeName:}" failed. No retries permitted until 2026-07-22 20:58:03.937948087 +0000 UTC m=+66.338750053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7mrf9" (UniqueName: "kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9") pod "network-check-target-fwwhl" (UID: "41af05a1-9431-4de7-8331-b7488704f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 20:57:32.947960 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.947773    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-38.ec2.internal" event="NodeReady"
Jul 22 20:57:32.948397 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.948059    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 20:57:32.987627 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.987604    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cw4qm"]
Jul 22 20:57:32.990494 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.990470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:32.992711 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.992690    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 20:57:32.992711 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.992702    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6fmcx\""
Jul 22 20:57:32.992879 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.992692    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 20:57:32.995449 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.995331    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mqn7j"]
Jul 22 20:57:32.997481 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.997457    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:32.999432 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.999414    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 20:57:32.999689 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.999671    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 20:57:32.999807 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.999690    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 20:57:32.999980 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:32.999965    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5ll7d\""
Jul 22 20:57:33.004449 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.004430    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cw4qm"]
Jul 22 20:57:33.011012 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.007324    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mqn7j"]
Jul 22 20:57:33.046300 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.046273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.046424 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.046314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10d8d735-5141-43ff-a8b9-6a35615d62b5-config-volume\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.046424 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.046341    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10d8d735-5141-43ff-a8b9-6a35615d62b5-tmp-dir\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.046530 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.046442    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvtnh\" (UniqueName: \"kubernetes.io/projected/ac9cb0ea-7a64-428e-b599-b13b45e763ce-kube-api-access-kvtnh\") pod \"ingress-canary-mqn7j\" (UID: \"ac9cb0ea-7a64-428e-b599-b13b45e763ce\") " pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:33.046530 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.046479    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac9cb0ea-7a64-428e-b599-b13b45e763ce-cert\") pod \"ingress-canary-mqn7j\" (UID: \"ac9cb0ea-7a64-428e-b599-b13b45e763ce\") " pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:33.046607 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.046570    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h7j2\" (UniqueName: \"kubernetes.io/projected/10d8d735-5141-43ff-a8b9-6a35615d62b5-kube-api-access-2h7j2\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.071433 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.071344    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:57:33.071533 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.071465    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:57:33.071592 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.071575    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:57:33.073796 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.073774    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 20:57:33.073896 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.073816    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 20:57:33.073896 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.073827    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 20:57:33.073896 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.073841    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 20:57:33.074059 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.073957    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j26qm\""
Jul 22 20:57:33.074193 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.074156    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bvpxr\""
Jul 22 20:57:33.105817 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.105791    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:33.146904 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.146883    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.147009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.146914    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10d8d735-5141-43ff-a8b9-6a35615d62b5-config-volume\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.147009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.146931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10d8d735-5141-43ff-a8b9-6a35615d62b5-tmp-dir\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.147009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.146966    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvtnh\" (UniqueName: \"kubernetes.io/projected/ac9cb0ea-7a64-428e-b599-b13b45e763ce-kube-api-access-kvtnh\") pod \"ingress-canary-mqn7j\" (UID: \"ac9cb0ea-7a64-428e-b599-b13b45e763ce\") " pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:33.147009 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.146992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac9cb0ea-7a64-428e-b599-b13b45e763ce-cert\") pod \"ingress-canary-mqn7j\" (UID: \"ac9cb0ea-7a64-428e-b599-b13b45e763ce\") " pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:33.147241 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:33.147037    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:57:33.147241 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:33.147098    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:33.647078854 +0000 UTC m=+36.047880819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:57:33.147241 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.147120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h7j2\" (UniqueName: \"kubernetes.io/projected/10d8d735-5141-43ff-a8b9-6a35615d62b5-kube-api-access-2h7j2\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.147449 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.147273    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10d8d735-5141-43ff-a8b9-6a35615d62b5-tmp-dir\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.147837 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.147811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/10d8d735-5141-43ff-a8b9-6a35615d62b5-config-volume\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.151673 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.151654    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac9cb0ea-7a64-428e-b599-b13b45e763ce-cert\") pod \"ingress-canary-mqn7j\" (UID: \"ac9cb0ea-7a64-428e-b599-b13b45e763ce\") " pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:33.158411 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.158390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvtnh\" (UniqueName: \"kubernetes.io/projected/ac9cb0ea-7a64-428e-b599-b13b45e763ce-kube-api-access-kvtnh\") pod \"ingress-canary-mqn7j\" (UID: \"ac9cb0ea-7a64-428e-b599-b13b45e763ce\") " pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:33.158522 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.158477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h7j2\" (UniqueName: \"kubernetes.io/projected/10d8d735-5141-43ff-a8b9-6a35615d62b5-kube-api-access-2h7j2\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.208432 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.208405    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:33.314815 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.314778    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mqn7j"
Jul 22 20:57:33.649946 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:33.649915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:33.650140 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:33.650084    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:57:33.650237 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:33.650169    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:34.650149236 +0000 UTC m=+37.050951198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:57:34.241326 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:34.241299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:34.344246 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:34.344221    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:34.351501 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:34.351465    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mqn7j"]
Jul 22 20:57:34.355062 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:57:34.355039    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9cb0ea_7a64_428e_b599_b13b45e763ce.slice/crio-15dcfc32a67072bf8e0d2f88d8200e59b03c0faaae66d4841e0ff2b0835ba87c WatchSource:0}: Error finding container 15dcfc32a67072bf8e0d2f88d8200e59b03c0faaae66d4841e0ff2b0835ba87c: Status 404 returned error can't find the container with id 15dcfc32a67072bf8e0d2f88d8200e59b03c0faaae66d4841e0ff2b0835ba87c
Jul 22 20:57:34.657511 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:34.657485    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:34.657621 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:34.657610    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:57:34.657691 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:34.657681    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:36.657666101 +0000 UTC m=+39.058468061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:57:35.347940 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:35.347913    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5" containerID="4a5eb14a38529ad81b37c3aba593eb65229e15d51662f2e3183aff5a530b1818" exitCode=0
Jul 22 20:57:35.348359 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:35.347994    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerDied","Data":"4a5eb14a38529ad81b37c3aba593eb65229e15d51662f2e3183aff5a530b1818"}
Jul 22 20:57:35.349247 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:35.349221    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mqn7j" event={"ID":"ac9cb0ea-7a64-428e-b599-b13b45e763ce","Type":"ContainerStarted","Data":"15dcfc32a67072bf8e0d2f88d8200e59b03c0faaae66d4841e0ff2b0835ba87c"}
Jul 22 20:57:36.358894 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:36.358657    2559 generic.go:356] "Generic (PLEG): container finished" podID="4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5" containerID="f9ffa2efdb5f8656838ff7a84ae754ebc9677524b21bc5ab06aaf5e5aacad817" exitCode=0
Jul 22 20:57:36.358894 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:36.358700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerDied","Data":"f9ffa2efdb5f8656838ff7a84ae754ebc9677524b21bc5ab06aaf5e5aacad817"}
Jul 22 20:57:36.671147 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:36.671121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:36.671276 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:36.671261    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:57:36.671336 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:36.671322    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:40.671302465 +0000 UTC m=+43.072104444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:57:37.362883 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:37.362788    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mqn7j" event={"ID":"ac9cb0ea-7a64-428e-b599-b13b45e763ce","Type":"ContainerStarted","Data":"bf30ed3e95724336b9afca577e633e4e7dd1427c981c94edc536bf1cc67fa1cc"}
Jul 22 20:57:37.366163 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:37.366137    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" event={"ID":"4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5","Type":"ContainerStarted","Data":"3cafd7d01173adf5bc60ed323209da657b0c43b4af30eda4273e809181852086"}
Jul 22 20:57:37.378901 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:37.378853    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mqn7j" podStartSLOduration=2.902459983 podStartE2EDuration="5.378835328s" totalImagesPullingTime="2.476375345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:37.378476417 +0000 UTC m=+39.779278402" watchObservedRunningTime="2026-07-22 20:57:37.378835328 +0000 UTC m=+39.779637312"
Jul 22 20:57:37.401063 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:37.401019    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xxtv6" podStartSLOduration=10.230478368 podStartE2EDuration="39.401001095s" totalImagesPullingTime="29.170522727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:57:37.400140969 +0000 UTC m=+39.800942984" watchObservedRunningTime="2026-07-22 20:57:37.401001095 +0000 UTC m=+39.801803078"
Jul 22 20:57:37.536972 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:37.536938    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:37.637825 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:37.637761    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:40.698511 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:40.698480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:40.698913 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:40.698575    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:57:40.698913 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:40.698620    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:57:48.69860764 +0000 UTC m=+51.099409601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:57:43.695794 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:43.695761    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:43.792360 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:43.792334    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:48.749929 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:48.749892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:57:48.750326 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:48.750029    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:57:48.750326 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:57:48.750095    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:58:04.750080059 +0000 UTC m=+67.150882023 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:57:59.336600 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:59.336567    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tbv4"
Jul 22 20:57:59.448892 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:59.448868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:57:59.550776 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:57:59.550743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:58:03.845116 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.845066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:58:03.847500 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.847482    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 20:58:03.859357 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.859332    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c7417f6-3df6-418d-b90c-5e7f49f92fe9-original-pull-secret\") pod \"global-pull-secret-syncer-8wfk7\" (UID: \"8c7417f6-3df6-418d-b90c-5e7f49f92fe9\") " pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:58:03.945586 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.945556    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:58:03.945714 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.945601    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:58:03.947793 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.947772    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 20:58:03.947943 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.947927    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 20:58:03.956489 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:58:03.956470    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 20:58:03.956570 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:58:03.956520    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 20:59:07.956506534 +0000 UTC m=+130.357308498 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : secret "metrics-daemon-secret" not found
Jul 22 20:58:03.958106 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.958089    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 20:58:03.968726 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.968700    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mrf9\" (UniqueName: \"kubernetes.io/projected/41af05a1-9431-4de7-8331-b7488704f7de-kube-api-access-7mrf9\") pod \"network-check-target-fwwhl\" (UID: \"41af05a1-9431-4de7-8331-b7488704f7de\") " pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:58:03.984592 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.984568    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bvpxr\""
Jul 22 20:58:03.993327 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.993309    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:58:03.995527 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:03.995513    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8wfk7"
Jul 22 20:58:04.117999 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.117937    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8wfk7"]
Jul 22 20:58:04.121109 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:58:04.121081    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7417f6_3df6_418d_b90c_5e7f49f92fe9.slice/crio-2412274c8c8bbd63e7de09b6d6b0caf2830004fd9fab6fb0d21dc0e9333c78a0 WatchSource:0}: Error finding container 2412274c8c8bbd63e7de09b6d6b0caf2830004fd9fab6fb0d21dc0e9333c78a0: Status 404 returned error can't find the container with id 2412274c8c8bbd63e7de09b6d6b0caf2830004fd9fab6fb0d21dc0e9333c78a0
Jul 22 20:58:04.135908 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.135887    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fwwhl"]
Jul 22 20:58:04.138707 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:58:04.138684    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41af05a1_9431_4de7_8331_b7488704f7de.slice/crio-1a285a1f27b0f8bcd8f9d8e531cac39c417923baf11c8756b5d11f3691a2fbaa WatchSource:0}: Error finding container 1a285a1f27b0f8bcd8f9d8e531cac39c417923baf11c8756b5d11f3691a2fbaa: Status 404 returned error can't find the container with id 1a285a1f27b0f8bcd8f9d8e531cac39c417923baf11c8756b5d11f3691a2fbaa
Jul 22 20:58:04.427986 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.427961    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8wfk7" event={"ID":"8c7417f6-3df6-418d-b90c-5e7f49f92fe9","Type":"ContainerStarted","Data":"2412274c8c8bbd63e7de09b6d6b0caf2830004fd9fab6fb0d21dc0e9333c78a0"}
Jul 22 20:58:04.428987 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.428964    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fwwhl" event={"ID":"41af05a1-9431-4de7-8331-b7488704f7de","Type":"ContainerStarted","Data":"1a285a1f27b0f8bcd8f9d8e531cac39c417923baf11c8756b5d11f3691a2fbaa"}
Jul 22 20:58:04.689079 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.688849    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"]
Jul 22 20:58:04.692290 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.692268    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"]
Jul 22 20:58:04.692435 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.692413    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:04.694693 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.694669    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5cvnf\""
Jul 22 20:58:04.695343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.695095    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 20:58:04.695343 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.695168    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 20:58:04.695509 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.695444    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 20:58:04.695566 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.695514    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 22 20:58:04.697498 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.697478    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.699843 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.699825    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 22 20:58:04.704617 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.703886    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"]
Jul 22 20:58:04.704810 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.704793    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"]
Jul 22 20:58:04.750982 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.750961    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/065c0ca8-8a3e-4a66-b951-f169fb009eb6-klusterlet-config\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.751090 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.750992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:58:04.751090 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.751017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4rhw\" (UniqueName: \"kubernetes.io/projected/065c0ca8-8a3e-4a66-b951-f169fb009eb6-kube-api-access-w4rhw\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.751216 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:58:04.751130    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:58:04.751216 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.751130    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kvjf\" (UniqueName: \"kubernetes.io/projected/007a162f-3edc-45ce-96f2-d49ce9eff50a-kube-api-access-9kvjf\") pod \"managed-serviceaccount-addon-agent-6d48b4f668-rrwwm\" (UID: \"007a162f-3edc-45ce-96f2-d49ce9eff50a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:04.751216 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:58:04.751205    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:58:36.751170453 +0000 UTC m=+99.151972420 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:58:04.751396 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.751248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/065c0ca8-8a3e-4a66-b951-f169fb009eb6-tmp\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.751396 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.751288    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/007a162f-3edc-45ce-96f2-d49ce9eff50a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d48b4f668-rrwwm\" (UID: \"007a162f-3edc-45ce-96f2-d49ce9eff50a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:04.852408 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.852382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/065c0ca8-8a3e-4a66-b951-f169fb009eb6-tmp\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.852822 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.852416    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/007a162f-3edc-45ce-96f2-d49ce9eff50a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d48b4f668-rrwwm\" (UID: \"007a162f-3edc-45ce-96f2-d49ce9eff50a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:04.852822 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.852491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/065c0ca8-8a3e-4a66-b951-f169fb009eb6-klusterlet-config\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.852822 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.852536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4rhw\" (UniqueName: \"kubernetes.io/projected/065c0ca8-8a3e-4a66-b951-f169fb009eb6-kube-api-access-w4rhw\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.852822 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.852565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kvjf\" (UniqueName: \"kubernetes.io/projected/007a162f-3edc-45ce-96f2-d49ce9eff50a-kube-api-access-9kvjf\") pod \"managed-serviceaccount-addon-agent-6d48b4f668-rrwwm\" (UID: \"007a162f-3edc-45ce-96f2-d49ce9eff50a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:04.853075 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.853048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/065c0ca8-8a3e-4a66-b951-f169fb009eb6-tmp\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.855389 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.855359    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/007a162f-3edc-45ce-96f2-d49ce9eff50a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d48b4f668-rrwwm\" (UID: \"007a162f-3edc-45ce-96f2-d49ce9eff50a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:04.855389 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.855377    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/065c0ca8-8a3e-4a66-b951-f169fb009eb6-klusterlet-config\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.860707 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.860685    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4rhw\" (UniqueName: \"kubernetes.io/projected/065c0ca8-8a3e-4a66-b951-f169fb009eb6-kube-api-access-w4rhw\") pod \"klusterlet-addon-workmgr-58b7987d59-56gg8\" (UID: \"065c0ca8-8a3e-4a66-b951-f169fb009eb6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:04.860825 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:04.860715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kvjf\" (UniqueName: \"kubernetes.io/projected/007a162f-3edc-45ce-96f2-d49ce9eff50a-kube-api-access-9kvjf\") pod \"managed-serviceaccount-addon-agent-6d48b4f668-rrwwm\" (UID: \"007a162f-3edc-45ce-96f2-d49ce9eff50a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:05.015140 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:05.015060    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 20:58:05.021244 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:05.021218    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:05.186827 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:05.186785    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"]
Jul 22 20:58:05.193395 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:58:05.193362    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod007a162f_3edc_45ce_96f2_d49ce9eff50a.slice/crio-6615e0414fa64f949a1d1e25724b44775e8652ece8531f43255ded267f564e0d WatchSource:0}: Error finding container 6615e0414fa64f949a1d1e25724b44775e8652ece8531f43255ded267f564e0d: Status 404 returned error can't find the container with id 6615e0414fa64f949a1d1e25724b44775e8652ece8531f43255ded267f564e0d
Jul 22 20:58:05.207780 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:05.207735    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"]
Jul 22 20:58:05.211086 ip-10-0-131-38 kubenswrapper[2559]: W0722 20:58:05.211059    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065c0ca8_8a3e_4a66_b951_f169fb009eb6.slice/crio-053b3f3dd578eed78840833ab455e7953a3e3e3cb387502b1d83d71dcf3e13bf WatchSource:0}: Error finding container 053b3f3dd578eed78840833ab455e7953a3e3e3cb387502b1d83d71dcf3e13bf: Status 404 returned error can't find the container with id 053b3f3dd578eed78840833ab455e7953a3e3e3cb387502b1d83d71dcf3e13bf
Jul 22 20:58:05.432894 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:05.432858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8" event={"ID":"065c0ca8-8a3e-4a66-b951-f169fb009eb6","Type":"ContainerStarted","Data":"053b3f3dd578eed78840833ab455e7953a3e3e3cb387502b1d83d71dcf3e13bf"}
Jul 22 20:58:05.434237 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:05.434207    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm" event={"ID":"007a162f-3edc-45ce-96f2-d49ce9eff50a","Type":"ContainerStarted","Data":"6615e0414fa64f949a1d1e25724b44775e8652ece8531f43255ded267f564e0d"}
Jul 22 20:58:09.445763 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:09.445722    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fwwhl" event={"ID":"41af05a1-9431-4de7-8331-b7488704f7de","Type":"ContainerStarted","Data":"800829821ea06e66af1f63457d39c2a02c4484e33f1d0b0c6a12d639c2c1910c"}
Jul 22 20:58:09.446239 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:09.445885    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:58:09.446998 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:09.446977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8wfk7" event={"ID":"8c7417f6-3df6-418d-b90c-5e7f49f92fe9","Type":"ContainerStarted","Data":"78550c9750f9ad6521dae39b25963093949596a8833c322e1083e84902f841b6"}
Jul 22 20:58:09.460803 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:09.460762    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fwwhl" podStartSLOduration=66.734260844 podStartE2EDuration="1m11.460751525s" totalImagesPullingTime="4.726490681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:58:09.460295849 +0000 UTC m=+71.861097832" watchObservedRunningTime="2026-07-22 20:58:09.460751525 +0000 UTC m=+71.861553508"
Jul 22 20:58:09.473534 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:09.473497    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8wfk7" podStartSLOduration=66.726038347 podStartE2EDuration="1m11.473482163s" totalImagesPullingTime="4.747443816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:58:09.473393118 +0000 UTC m=+71.874195114" watchObservedRunningTime="2026-07-22 20:58:09.473482163 +0000 UTC m=+71.874284128"
Jul 22 20:58:25.488553 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:25.488516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm" event={"ID":"007a162f-3edc-45ce-96f2-d49ce9eff50a","Type":"ContainerStarted","Data":"fc48ef375a0dd47f324069a3c6d4166231787466cfd16044228268a5b03a1426"}
Jul 22 20:58:25.503748 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:25.503689    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm" podStartSLOduration=2.027654924 podStartE2EDuration="21.503670306s" totalImagesPullingTime="19.476015382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:58:25.502469908 +0000 UTC m=+87.903271914" watchObservedRunningTime="2026-07-22 20:58:25.503670306 +0000 UTC m=+87.904472289"
Jul 22 20:58:26.492685 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:26.492649    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8" event={"ID":"065c0ca8-8a3e-4a66-b951-f169fb009eb6","Type":"ContainerStarted","Data":"e2b973bde3dbf2765619c9130140b4e2dea4e123896efcc287358e30acee48e6"}
Jul 22 20:58:26.509372 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:26.509333    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8" podStartSLOduration=1.5943977660000002 podStartE2EDuration="22.509318212s" totalImagesPullingTime="20.914920446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 20:58:26.508568928 +0000 UTC m=+88.909370922" watchObservedRunningTime="2026-07-22 20:58:26.509318212 +0000 UTC m=+88.910120218"
Jul 22 20:58:26.789193 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:26.789117    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:58:26.868674 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:26.868652    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 20:58:27.496238 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:27.496212    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:27.497701 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:27.497684    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 20:58:36.758137 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:36.758105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:58:36.758516 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:58:36.758221    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:58:36.758516 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:58:36.758270    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 20:59:40.758257859 +0000 UTC m=+163.159059821 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:58:40.452310 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:58:40.452283    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fwwhl"
Jul 22 20:59:07.965481 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:59:07.965434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 20:59:07.965958 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:59:07.965581    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 20:59:07.965958 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:59:07.965656    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:09.965639924 +0000 UTC m=+252.366441885 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : secret "metrics-daemon-secret" not found
Jul 22 20:59:36.002397 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:59:36.002349    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cw4qm" podUID="10d8d735-5141-43ff-a8b9-6a35615d62b5"
Jul 22 20:59:36.089849 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:59:36.089821    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 20:59:36.685262 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:59:36.685231    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:59:40.780267 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:59:40.780234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 20:59:40.780819 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:59:40.780350    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 20:59:40.780819 ip-10-0-131-38 kubenswrapper[2559]: E0722 20:59:40.780436    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls podName:10d8d735-5141-43ff-a8b9-6a35615d62b5 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:42.780416061 +0000 UTC m=+285.181218021 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls") pod "dns-default-cw4qm" (UID: "10d8d735-5141-43ff-a8b9-6a35615d62b5") : secret "dns-default-metrics-tls" not found
Jul 22 20:59:48.072260 ip-10-0-131-38 kubenswrapper[2559]: I0722 20:59:48.072225    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 21:00:24.816993 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:24.816968    2559 generic.go:356] "Generic (PLEG): container finished" podID="007a162f-3edc-45ce-96f2-d49ce9eff50a" containerID="fc48ef375a0dd47f324069a3c6d4166231787466cfd16044228268a5b03a1426" exitCode=255
Jul 22 21:00:24.817408 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:24.817005    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm" event={"ID":"007a162f-3edc-45ce-96f2-d49ce9eff50a","Type":"ContainerDied","Data":"fc48ef375a0dd47f324069a3c6d4166231787466cfd16044228268a5b03a1426"}
Jul 22 21:00:24.817408 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:24.817367    2559 scope.go:117] "RemoveContainer" containerID="fc48ef375a0dd47f324069a3c6d4166231787466cfd16044228268a5b03a1426"
Jul 22 21:00:25.015554 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:25.015526    2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm"
Jul 22 21:00:25.821137 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:25.821105    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d48b4f668-rrwwm" event={"ID":"007a162f-3edc-45ce-96f2-d49ce9eff50a","Type":"ContainerStarted","Data":"a861e9c1c762419590ef2fbd526661ec061909956a56dfd4798bc134e646b0ae"}
Jul 22 21:00:26.825393 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:26.825366    2559 generic.go:356] "Generic (PLEG): container finished" podID="065c0ca8-8a3e-4a66-b951-f169fb009eb6" containerID="e2b973bde3dbf2765619c9130140b4e2dea4e123896efcc287358e30acee48e6" exitCode=1
Jul 22 21:00:26.825807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:26.825446    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8" event={"ID":"065c0ca8-8a3e-4a66-b951-f169fb009eb6","Type":"ContainerDied","Data":"e2b973bde3dbf2765619c9130140b4e2dea4e123896efcc287358e30acee48e6"}
Jul 22 21:00:26.825807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:26.825787    2559 scope.go:117] "RemoveContainer" containerID="e2b973bde3dbf2765619c9130140b4e2dea4e123896efcc287358e30acee48e6"
Jul 22 21:00:27.496685 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:27.496650    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 21:00:27.830113 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:27.830051    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8" event={"ID":"065c0ca8-8a3e-4a66-b951-f169fb009eb6","Type":"ContainerStarted","Data":"e4092e551402f74f167a18507527c9dbd53feea2fae9cf30ea6a4dfe9dd73b0b"}
Jul 22 21:00:27.830435 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:27.830265    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 21:00:27.830816 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:27.830799    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-58b7987d59-56gg8"
Jul 22 21:00:38.610616 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:00:38.610577    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:05.436168 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.436136    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"]
Jul 22 21:01:05.438253 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.438225    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"
Jul 22 21:01:05.438519 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.438486    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"]
Jul 22 21:01:05.440210 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.440191    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hsbgp\""
Jul 22 21:01:05.440355 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.440339    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"
Jul 22 21:01:05.440853 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.440834    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 22 21:01:05.440928 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.440844    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 22 21:01:05.442614 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.442597    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"]
Jul 22 21:01:05.444268 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.444243    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"]
Jul 22 21:01:05.444353 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.444338    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.445400 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.445379    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r9hnn\""
Jul 22 21:01:05.446466 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.446447    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-q2m6m\""
Jul 22 21:01:05.446568 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.446472    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 22 21:01:05.446568 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.446495    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 22 21:01:05.446568 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.446506    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:05.446568 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.446538    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 22 21:01:05.446795 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.446741    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 22 21:01:05.448958 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.448939    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 22 21:01:05.449193 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.449165    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 22 21:01:05.449274 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.449229    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 22 21:01:05.449327 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.449287    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"]
Jul 22 21:01:05.449364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.449350    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-85rgp\""
Jul 22 21:01:05.451742 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.451723    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"]
Jul 22 21:01:05.456213 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.456141    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"]
Jul 22 21:01:05.456902 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.456885    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"]
Jul 22 21:01:05.517052 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.517028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:05.517153 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.517060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbv47\" (UniqueName: \"kubernetes.io/projected/0c0c8c06-e310-400c-8a0b-be33755f5f79-kube-api-access-dbv47\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.517153 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.517144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c0c8c06-e310-400c-8a0b-be33755f5f79-config\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.517301 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.517206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzrmc\" (UniqueName: \"kubernetes.io/projected/42c52735-5a72-47dd-93a7-49b2185ab990-kube-api-access-mzrmc\") pod \"network-check-source-7bc5b6bf85-28cjp\" (UID: \"42c52735-5a72-47dd-93a7-49b2185ab990\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"
Jul 22 21:01:05.517301 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.517231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c0c8c06-e310-400c-8a0b-be33755f5f79-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.517301 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.517268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkmw2\" (UniqueName: \"kubernetes.io/projected/ffff2ea6-b0d9-4adc-8b92-ce54ca376455-kube-api-access-bkmw2\") pod \"volume-data-source-validator-78cf4c6f6-tnwdm\" (UID: \"ffff2ea6-b0d9-4adc-8b92-ce54ca376455\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"
Jul 22 21:01:05.517301 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.517294    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htrsm\" (UniqueName: \"kubernetes.io/projected/537a3a27-179f-40a0-8943-df6a86a9e720-kube-api-access-htrsm\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:05.577221 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.577196    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:05.618333 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c0c8c06-e310-400c-8a0b-be33755f5f79-config\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.618409 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618345    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzrmc\" (UniqueName: \"kubernetes.io/projected/42c52735-5a72-47dd-93a7-49b2185ab990-kube-api-access-mzrmc\") pod \"network-check-source-7bc5b6bf85-28cjp\" (UID: \"42c52735-5a72-47dd-93a7-49b2185ab990\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"
Jul 22 21:01:05.618409 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c0c8c06-e310-400c-8a0b-be33755f5f79-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.618482 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkmw2\" (UniqueName: \"kubernetes.io/projected/ffff2ea6-b0d9-4adc-8b92-ce54ca376455-kube-api-access-bkmw2\") pod \"volume-data-source-validator-78cf4c6f6-tnwdm\" (UID: \"ffff2ea6-b0d9-4adc-8b92-ce54ca376455\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"
Jul 22 21:01:05.618482 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618442    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htrsm\" (UniqueName: \"kubernetes.io/projected/537a3a27-179f-40a0-8943-df6a86a9e720-kube-api-access-htrsm\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:05.618588 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:05.618645 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618609    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbv47\" (UniqueName: \"kubernetes.io/projected/0c0c8c06-e310-400c-8a0b-be33755f5f79-kube-api-access-dbv47\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.618708 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:05.618688    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 21:01:05.618757 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:05.618740    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls podName:537a3a27-179f-40a0-8943-df6a86a9e720 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:06.11872525 +0000 UTC m=+248.519527221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hnnvl" (UID: "537a3a27-179f-40a0-8943-df6a86a9e720") : secret "samples-operator-tls" not found
Jul 22 21:01:05.618982 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.618964    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c0c8c06-e310-400c-8a0b-be33755f5f79-config\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.620526 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.620502    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c0c8c06-e310-400c-8a0b-be33755f5f79-serving-cert\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.626745 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.626718    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzrmc\" (UniqueName: \"kubernetes.io/projected/42c52735-5a72-47dd-93a7-49b2185ab990-kube-api-access-mzrmc\") pod \"network-check-source-7bc5b6bf85-28cjp\" (UID: \"42c52735-5a72-47dd-93a7-49b2185ab990\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"
Jul 22 21:01:05.626828 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.626787    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbv47\" (UniqueName: \"kubernetes.io/projected/0c0c8c06-e310-400c-8a0b-be33755f5f79-kube-api-access-dbv47\") pod \"service-ca-operator-5bcbc94c78-hq467\" (UID: \"0c0c8c06-e310-400c-8a0b-be33755f5f79\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.627392 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.627371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkmw2\" (UniqueName: \"kubernetes.io/projected/ffff2ea6-b0d9-4adc-8b92-ce54ca376455-kube-api-access-bkmw2\") pod \"volume-data-source-validator-78cf4c6f6-tnwdm\" (UID: \"ffff2ea6-b0d9-4adc-8b92-ce54ca376455\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"
Jul 22 21:01:05.627441 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.627374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htrsm\" (UniqueName: \"kubernetes.io/projected/537a3a27-179f-40a0-8943-df6a86a9e720-kube-api-access-htrsm\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:05.675387 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.675361    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:05.748360 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.748316    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"
Jul 22 21:01:05.754974 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.754958    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"
Jul 22 21:01:05.763022 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.763004    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"
Jul 22 21:01:05.887748 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:05.887725    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffff2ea6_b0d9_4adc_8b92_ce54ca376455.slice/crio-27994489e87a2b634ff7f5cd5792b68153b202f1f6ce10f081b6a713eb3af548 WatchSource:0}: Error finding container 27994489e87a2b634ff7f5cd5792b68153b202f1f6ce10f081b6a713eb3af548: Status 404 returned error can't find the container with id 27994489e87a2b634ff7f5cd5792b68153b202f1f6ce10f081b6a713eb3af548
Jul 22 21:01:05.889350 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.889315    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm"]
Jul 22 21:01:05.934635 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:05.934608    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm" event={"ID":"ffff2ea6-b0d9-4adc-8b92-ce54ca376455","Type":"ContainerStarted","Data":"27994489e87a2b634ff7f5cd5792b68153b202f1f6ce10f081b6a713eb3af548"}
Jul 22 21:01:06.110493 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:06.110435    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp"]
Jul 22 21:01:06.112981 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:06.112959    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467"]
Jul 22 21:01:06.113573 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:06.113553    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c52735_5a72_47dd_93a7_49b2185ab990.slice/crio-090fa1ef1d08a3725a60c8aef764052a1c4f86c7f3a2372f5ed410cf3c85b4dd WatchSource:0}: Error finding container 090fa1ef1d08a3725a60c8aef764052a1c4f86c7f3a2372f5ed410cf3c85b4dd: Status 404 returned error can't find the container with id 090fa1ef1d08a3725a60c8aef764052a1c4f86c7f3a2372f5ed410cf3c85b4dd
Jul 22 21:01:06.116325 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:06.116299    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0c8c06_e310_400c_8a0b_be33755f5f79.slice/crio-08472830cd9df3abc2f8a4be41cf16284ed9cfe78a823f797ef3dbf06678cb10 WatchSource:0}: Error finding container 08472830cd9df3abc2f8a4be41cf16284ed9cfe78a823f797ef3dbf06678cb10: Status 404 returned error can't find the container with id 08472830cd9df3abc2f8a4be41cf16284ed9cfe78a823f797ef3dbf06678cb10
Jul 22 21:01:06.121637 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:06.121618    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:06.121723 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:06.121715    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 21:01:06.121772 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:06.121763    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls podName:537a3a27-179f-40a0-8943-df6a86a9e720 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:07.121746748 +0000 UTC m=+249.522548723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hnnvl" (UID: "537a3a27-179f-40a0-8943-df6a86a9e720") : secret "samples-operator-tls" not found
Jul 22 21:01:06.939080 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:06.938905    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467" event={"ID":"0c0c8c06-e310-400c-8a0b-be33755f5f79","Type":"ContainerStarted","Data":"08472830cd9df3abc2f8a4be41cf16284ed9cfe78a823f797ef3dbf06678cb10"}
Jul 22 21:01:06.944499 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:06.944467    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp" event={"ID":"42c52735-5a72-47dd-93a7-49b2185ab990","Type":"ContainerStarted","Data":"1b072a8d401e9ff8f01aa9bc79e5890c4b62b802349fcf495c79ba81239ae2f8"}
Jul 22 21:01:06.944663 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:06.944516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp" event={"ID":"42c52735-5a72-47dd-93a7-49b2185ab990","Type":"ContainerStarted","Data":"090fa1ef1d08a3725a60c8aef764052a1c4f86c7f3a2372f5ed410cf3c85b4dd"}
Jul 22 21:01:06.959488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:06.959435    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-28cjp" podStartSLOduration=1.959416665 podStartE2EDuration="1.959416665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:01:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:06.958033384 +0000 UTC m=+249.358835480" watchObservedRunningTime="2026-07-22 21:01:06.959416665 +0000 UTC m=+249.360218649"
Jul 22 21:01:07.130996 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:07.130953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:07.131172 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:07.131121    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 21:01:07.131265 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:07.131206    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls podName:537a3a27-179f-40a0-8943-df6a86a9e720 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:09.131171251 +0000 UTC m=+251.531973231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hnnvl" (UID: "537a3a27-179f-40a0-8943-df6a86a9e720") : secret "samples-operator-tls" not found
Jul 22 21:01:07.948895 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:07.948858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm" event={"ID":"ffff2ea6-b0d9-4adc-8b92-ce54ca376455","Type":"ContainerStarted","Data":"015096eacb07780be09859ec820741dbffc048fb19c9d25dc2c438d9ed43efac"}
Jul 22 21:01:07.964790 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:07.964740    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-tnwdm" podStartSLOduration=1.6177405299999998 podStartE2EDuration="2.964691713s" totalImagesPullingTime="1.346951183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:07.963292438 +0000 UTC m=+250.364094422" watchObservedRunningTime="2026-07-22 21:01:07.964691713 +0000 UTC m=+250.365493676"
Jul 22 21:01:08.047841 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:08.047808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:08.952846 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:08.952807    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467" event={"ID":"0c0c8c06-e310-400c-8a0b-be33755f5f79","Type":"ContainerStarted","Data":"4d2106c943afa820a01ee10102d3642e1b78af187dcdb13a87f8cc65c41b244f"}
Jul 22 21:01:08.969771 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:08.969719    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467" podStartSLOduration=1.6472954990000002 podStartE2EDuration="3.969700611s" totalImagesPullingTime="2.322405112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:08.968462411 +0000 UTC m=+251.369264394" watchObservedRunningTime="2026-07-22 21:01:08.969700611 +0000 UTC m=+251.370502596"
Jul 22 21:01:09.144228 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.144199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:09.144364 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:09.144319    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 21:01:09.144406 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:09.144372    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls podName:537a3a27-179f-40a0-8943-df6a86a9e720 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:13.144357708 +0000 UTC m=+255.545159673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hnnvl" (UID: "537a3a27-179f-40a0-8943-df6a86a9e720") : secret "samples-operator-tls" not found
Jul 22 21:01:09.562096 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.562061    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:09.614280 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.614253    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"]
Jul 22 21:01:09.616478 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.616463    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"
Jul 22 21:01:09.618703 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.618685    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 22 21:01:09.619318 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.619301    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 22 21:01:09.619407 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.619334    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8plvb\""
Jul 22 21:01:09.625374 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.625357    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"]
Jul 22 21:01:09.645191 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.645155    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:09.696223 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.696199    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:09.748225 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.748196    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5hzn\" (UniqueName: \"kubernetes.io/projected/c713457b-8861-4ce3-97c8-d6a8d6c73670-kube-api-access-p5hzn\") pod \"migrator-7cc9c7ffd7-479sj\" (UID: \"c713457b-8861-4ce3-97c8-d6a8d6c73670\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"
Jul 22 21:01:09.797004 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.796980    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:09.848625 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.848569    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5hzn\" (UniqueName: \"kubernetes.io/projected/c713457b-8861-4ce3-97c8-d6a8d6c73670-kube-api-access-p5hzn\") pod \"migrator-7cc9c7ffd7-479sj\" (UID: \"c713457b-8861-4ce3-97c8-d6a8d6c73670\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"
Jul 22 21:01:09.856451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.856432    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5hzn\" (UniqueName: \"kubernetes.io/projected/c713457b-8861-4ce3-97c8-d6a8d6c73670-kube-api-access-p5hzn\") pod \"migrator-7cc9c7ffd7-479sj\" (UID: \"c713457b-8861-4ce3-97c8-d6a8d6c73670\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"
Jul 22 21:01:09.860785 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.860765    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:09.925066 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:09.925039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"
Jul 22 21:01:10.039017 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:10.038989    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj"]
Jul 22 21:01:10.041781 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:10.041747    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc713457b_8861_4ce3_97c8_d6a8d6c73670.slice/crio-b89bf31d47880dcb87b6c811d590258710b7bfe76ba641b4d86f05fe6e3fbdb9 WatchSource:0}: Error finding container b89bf31d47880dcb87b6c811d590258710b7bfe76ba641b4d86f05fe6e3fbdb9: Status 404 returned error can't find the container with id b89bf31d47880dcb87b6c811d590258710b7bfe76ba641b4d86f05fe6e3fbdb9
Jul 22 21:01:10.050355 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:10.050333    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 21:01:10.050477 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:10.050461    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 21:01:10.050525 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:10.050515    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs podName:4df97b04-40ba-466d-a735-cd1184db9185 nodeName:}" failed. No retries permitted until 2026-07-22 21:03:12.050496772 +0000 UTC m=+374.451298733 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs") pod "network-metrics-daemon-b62h4" (UID: "4df97b04-40ba-466d-a735-cd1184db9185") : secret "metrics-daemon-secret" not found
Jul 22 21:01:10.243509 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:10.243471    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:10.344494 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:10.344468    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:10.962999 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:10.962964    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj" event={"ID":"c713457b-8861-4ce3-97c8-d6a8d6c73670","Type":"ContainerStarted","Data":"b89bf31d47880dcb87b6c811d590258710b7bfe76ba641b4d86f05fe6e3fbdb9"}
Jul 22 21:01:11.886497 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:11.886465    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:11.938951 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:11.938930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:12.050610 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.050586    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dmv75"]
Jul 22 21:01:12.053593 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.053576    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.056057 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.056039    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 21:01:12.056692 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.056667    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 21:01:12.056692 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.056679    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jvwv8\""
Jul 22 21:01:12.056840 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.056669    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 21:01:12.056840 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.056726    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 21:01:12.063493 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.063473    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dmv75"]
Jul 22 21:01:12.086700 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.086675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:12.162877 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.162858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9qrs\" (UniqueName: \"kubernetes.io/projected/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-api-access-f9qrs\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.162971 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.162911    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.162971 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.162963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e9c6314-dd34-4faa-abfb-e78a50ab275b-data-volume\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.163098 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.163080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e9c6314-dd34-4faa-abfb-e78a50ab275b-crio-socket\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.163168 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.163150    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.264318 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.264297    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.264403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.264326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9qrs\" (UniqueName: \"kubernetes.io/projected/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-api-access-f9qrs\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.264403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.264349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.264403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.264376    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e9c6314-dd34-4faa-abfb-e78a50ab275b-data-volume\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.264505 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.264412    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e9c6314-dd34-4faa-abfb-e78a50ab275b-crio-socket\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.264505 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.264426    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:12.264505 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.264438    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 21:01:12.264505 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.264479    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:12.764461797 +0000 UTC m=+255.165263759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:12.264505 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.264491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e9c6314-dd34-4faa-abfb-e78a50ab275b-crio-socket\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.264505 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.264502    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:12.764487784 +0000 UTC m=+255.165289754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : configmap "kube-rbac-proxy" not found
Jul 22 21:01:12.264747 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.264730    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e9c6314-dd34-4faa-abfb-e78a50ab275b-data-volume\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.275424 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.275400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9qrs\" (UniqueName: \"kubernetes.io/projected/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-api-access-f9qrs\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.767336 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.767310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.767475 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.767380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:12.767475 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.767421    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 21:01:12.767553 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.767478    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:12.767553 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.767486    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:13.767471887 +0000 UTC m=+256.168273848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : configmap "kube-rbac-proxy" not found
Jul 22 21:01:12.767553 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:12.767541    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:13.767526426 +0000 UTC m=+256.168328451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:12.845923 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:12.845898    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:13.170530 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:13.170504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:13.170821 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:13.170604    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 21:01:13.170821 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:13.170648    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls podName:537a3a27-179f-40a0-8943-df6a86a9e720 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:21.170636973 +0000 UTC m=+263.571438935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-hnnvl" (UID: "537a3a27-179f-40a0-8943-df6a86a9e720") : secret "samples-operator-tls" not found
Jul 22 21:01:13.774681 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:13.774649    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:13.774846 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:13.774751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:13.774846 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:13.774789    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 21:01:13.774964 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:13.774856    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:15.77483887 +0000 UTC m=+258.175640831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : configmap "kube-rbac-proxy" not found
Jul 22 21:01:13.774964 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:13.774859    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:13.774964 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:13.774919    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:15.774903253 +0000 UTC m=+258.175705216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:14.730462 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:14.730440    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7cqsd_a0b7daf0-ad29-4a64-916c-959616657b02/dns-node-resolver/0.log"
Jul 22 21:01:14.976995 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:14.976968    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj" event={"ID":"c713457b-8861-4ce3-97c8-d6a8d6c73670","Type":"ContainerStarted","Data":"c0884ab82290216f6b67b01fcb9fbe6fe9741791000dc7f145a3f39ad91cd090"}
Jul 22 21:01:14.977109 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:14.977029    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj" event={"ID":"c713457b-8861-4ce3-97c8-d6a8d6c73670","Type":"ContainerStarted","Data":"6b5291411f6408fbb6dd5ad2066325145d767641521b1eed81b6f49c6c78c7c7"}
Jul 22 21:01:14.992956 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:14.992889    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-479sj" podStartSLOduration=2.060820948 podStartE2EDuration="5.992875079s" totalImagesPullingTime="3.932054131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:14.992346068 +0000 UTC m=+257.393148052" watchObservedRunningTime="2026-07-22 21:01:14.992875079 +0000 UTC m=+257.393677061"
Jul 22 21:01:15.316012 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:15.315954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:15.416932 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:15.416905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:15.786823 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:15.786802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:15.787169 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:15.786866    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:15.787169 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:15.786919    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 21:01:15.787169 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:15.786973    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:15.787169 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:15.786980    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:19.786966085 +0000 UTC m=+262.187768049 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : configmap "kube-rbac-proxy" not found
Jul 22 21:01:15.787169 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:15.787042    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:19.787022224 +0000 UTC m=+262.187824199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:16.138616 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:16.138552    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g2qc8_2e752280-465f-4e6a-83d0-52251da4bc17/node-ca/0.log"
Jul 22 21:01:16.931190 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:16.931144    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mqn7j_ac9cb0ea-7a64-428e-b599-b13b45e763ce/serve-healthcheck-canary/0.log"
Jul 22 21:01:17.134400 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:17.134373    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-479sj_c713457b-8861-4ce3-97c8-d6a8d6c73670/migrator/0.log"
Jul 22 21:01:17.330933 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:17.330869    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-479sj_c713457b-8861-4ce3-97c8-d6a8d6c73670/graceful-termination/0.log"
Jul 22 21:01:18.235566 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:18.235534    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:18.338063 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:18.338030    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:18.614806 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:18.614739    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:19.256694 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:19.256666    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:19.383062 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:19.383036    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:19.441654 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:19.441636    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:19.620206 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:19.620127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:19.814414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:19.814391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:19.814492 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:19.814456    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:19.814546 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:19.814493    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 21:01:19.814546 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:19.814537    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:19.814546 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:19.814544    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:27.814531254 +0000 UTC m=+270.215333215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : configmap "kube-rbac-proxy" not found
Jul 22 21:01:19.814657 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:19.814586    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls podName:4e9c6314-dd34-4faa-abfb-e78a50ab275b nodeName:}" failed. No retries permitted until 2026-07-22 21:01:27.814575446 +0000 UTC m=+270.215377407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dmv75" (UID: "4e9c6314-dd34-4faa-abfb-e78a50ab275b") : secret "insights-runtime-extractor-tls" not found
Jul 22 21:01:20.024658 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:20.024585    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:20.078412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:20.078389    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:20.228068 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:20.228047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:20.817373 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:20.817344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:21.211734 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.211705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:21.226313 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.226294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:21.228500 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.228472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/537a3a27-179f-40a0-8943-df6a86a9e720-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-hnnvl\" (UID: \"537a3a27-179f-40a0-8943-df6a86a9e720\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:21.367152 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.367132    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"
Jul 22 21:01:21.481877 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.481817    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl"]
Jul 22 21:01:21.613028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.613002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:21.978465 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.978433    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:21.995476 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:21.995451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl" event={"ID":"537a3a27-179f-40a0-8943-df6a86a9e720","Type":"ContainerStarted","Data":"c2cb71562d4f1090d3838d712e08e4180614fa3fb3691f4f045cd9bfc7a2af76"}
Jul 22 21:01:22.075318 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:22.075295    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:22.208516 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:22.208491    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:22.414711 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:22.414674    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:22.800956 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:22.800873    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:23.420327 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:23.420286    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:23.813128 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:23.813093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:24.003260 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.003166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl" event={"ID":"537a3a27-179f-40a0-8943-df6a86a9e720","Type":"ContainerStarted","Data":"8bb9020724ef608838a961008cbcc21d682a8e9a89659677756444717641350f"}
Jul 22 21:01:24.003388 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.003273    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl" event={"ID":"537a3a27-179f-40a0-8943-df6a86a9e720","Type":"ContainerStarted","Data":"a24230b74839437c000f10a08e800c22965601fc14b55371e221b0e7614ab8aa"}
Jul 22 21:01:24.022285 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.022242    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-hnnvl" podStartSLOduration=16.865228529 podStartE2EDuration="19.022227483s" totalImagesPullingTime="2.156998954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:24.021229915 +0000 UTC m=+266.422031917" watchObservedRunningTime="2026-07-22 21:01:24.022227483 +0000 UTC m=+266.423029508"
Jul 22 21:01:24.025864 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.025813    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:24.212939 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.212905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:24.311972 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.311913    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:24.362602 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.362575    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:24.614327 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.614249    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:24.806702 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:24.806675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:25.426544 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:25.426508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:25.614151 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:25.614118    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:25.812364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:25.812283    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:26.023828 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:26.023795    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:27.874718 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:27.874688    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:27.875088 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:27.874728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:27.875196 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:27.875160    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e9c6314-dd34-4faa-abfb-e78a50ab275b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:27.876911 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:27.876889    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e9c6314-dd34-4faa-abfb-e78a50ab275b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dmv75\" (UID: \"4e9c6314-dd34-4faa-abfb-e78a50ab275b\") " pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:27.962215 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:27.962192    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dmv75"
Jul 22 21:01:28.085384 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:28.085361    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dmv75"]
Jul 22 21:01:28.087935 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:28.087914    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9c6314_dd34_4faa_abfb_e78a50ab275b.slice/crio-2cbe3be9eb45758e7521db6ecb8fb955efdfcda5b526183c047719531a3ebce3 WatchSource:0}: Error finding container 2cbe3be9eb45758e7521db6ecb8fb955efdfcda5b526183c047719531a3ebce3: Status 404 returned error can't find the container with id 2cbe3be9eb45758e7521db6ecb8fb955efdfcda5b526183c047719531a3ebce3
Jul 22 21:01:29.020012 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:29.019972    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmv75" event={"ID":"4e9c6314-dd34-4faa-abfb-e78a50ab275b","Type":"ContainerStarted","Data":"2cbe3be9eb45758e7521db6ecb8fb955efdfcda5b526183c047719531a3ebce3"}
Jul 22 21:01:33.035528 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.035497    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmv75" event={"ID":"4e9c6314-dd34-4faa-abfb-e78a50ab275b","Type":"ContainerStarted","Data":"73e9ba0d8265bf54092affd8d984aea536ab56faa0c4d8989454b095400ecc90"}
Jul 22 21:01:33.526842 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.526800    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-856988977b-txn6q"]
Jul 22 21:01:33.529494 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.529418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.532145 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.532111    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 21:01:33.532145 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.532127    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 21:01:33.532489 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.532471    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b7lth\""
Jul 22 21:01:33.533170 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.533010    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 21:01:33.541386 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.539244    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 21:01:33.545380 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.545358    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-856988977b-txn6q"]
Jul 22 21:01:33.549616 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.549542    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:33.617737 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-bound-sa-token\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.617878 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-image-registry-private-configuration\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.617878 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617767    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-trusted-ca\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.617878 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-installation-pull-secrets\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.617878 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617854    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-registry-certificates\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.618094 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-ca-trust-extracted\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.618094 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617923    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xwz\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-kube-api-access-f8xwz\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.618094 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.617978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-registry-tls\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.647613 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.647508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:33.718820 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.718755    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-installation-pull-secrets\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.718820 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.718807    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-registry-certificates\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.719024 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.718852    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-ca-trust-extracted\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.719024 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.718903    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xwz\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-kube-api-access-f8xwz\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.719024 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.718957    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-registry-tls\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.719024 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.718984    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-bound-sa-token\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.719024 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.719013    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-image-registry-private-configuration\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.720348 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.719037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-trusted-ca\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.720348 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.719841    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-ca-trust-extracted\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.720348 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.720005    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-trusted-ca\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.720712 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.720689    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-registry-certificates\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.723344 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.722826    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-installation-pull-secrets\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.723344 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.723297    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-registry-tls\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.725585 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.725559    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-image-registry-private-configuration\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.736935 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.736909    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xwz\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-kube-api-access-f8xwz\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.740610 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.740587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9-bound-sa-token\") pod \"image-registry-856988977b-txn6q\" (UID: \"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9\") " pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.842923 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.842858    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:33.987901 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:33.987868    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-856988977b-txn6q"]
Jul 22 21:01:33.990778 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:33.990751    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726d5fd7_3672_46ad_b3a4_fa2f3d8d40b9.slice/crio-de46c0537390c5e3f607b49bf4d1d153d15d9b671a82ea33e02ea69405a74759 WatchSource:0}: Error finding container de46c0537390c5e3f607b49bf4d1d153d15d9b671a82ea33e02ea69405a74759: Status 404 returned error can't find the container with id de46c0537390c5e3f607b49bf4d1d153d15d9b671a82ea33e02ea69405a74759
Jul 22 21:01:34.040675 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:34.040648    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-856988977b-txn6q" event={"ID":"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9","Type":"ContainerStarted","Data":"de46c0537390c5e3f607b49bf4d1d153d15d9b671a82ea33e02ea69405a74759"}
Jul 22 21:01:35.047399 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:35.047358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-856988977b-txn6q" event={"ID":"726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9","Type":"ContainerStarted","Data":"5458ed7c46da83f73ee167a7e7a79cb80bb7e563c8cdbf6a4471ef1c04b2b9d8"}
Jul 22 21:01:35.047841 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:35.047513    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:35.065668 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:35.065610    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-856988977b-txn6q" podStartSLOduration=2.065597672 podStartE2EDuration="2.065597672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:35.065345945 +0000 UTC m=+277.466147924" watchObservedRunningTime="2026-07-22 21:01:35.065597672 +0000 UTC m=+277.466399633"
Jul 22 21:01:37.057152 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:37.057117    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmv75" event={"ID":"4e9c6314-dd34-4faa-abfb-e78a50ab275b","Type":"ContainerStarted","Data":"1f8183ec2c204d8c646f463df06577efc9212eee50153cfd65b043aed06849a2"}
Jul 22 21:01:39.686577 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:39.686535    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cw4qm" podUID="10d8d735-5141-43ff-a8b9-6a35615d62b5"
Jul 22 21:01:40.068451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:40.068396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cw4qm"
Jul 22 21:01:40.068451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:40.068377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dmv75" event={"ID":"4e9c6314-dd34-4faa-abfb-e78a50ab275b","Type":"ContainerStarted","Data":"5ade0b0c9eda9ff160a30926c56bfa85ebbfe9e1b5b54b54f87774f8d7a803d9"}
Jul 22 21:01:40.087791 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:40.087753    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dmv75" podStartSLOduration=16.520502198 podStartE2EDuration="28.087740776s" totalImagesPullingTime="11.567238578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 21:01:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:40.086636525 +0000 UTC m=+282.487438510" watchObservedRunningTime="2026-07-22 21:01:40.087740776 +0000 UTC m=+282.488542758"
Jul 22 21:01:40.327313 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:40.327259    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:40.491155 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:40.491130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:41.026512 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:41.026483    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:42.783395 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:42.783368    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 21:01:42.785538 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:42.785517    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/10d8d735-5141-43ff-a8b9-6a35615d62b5-metrics-tls\") pod \"dns-default-cw4qm\" (UID: \"10d8d735-5141-43ff-a8b9-6a35615d62b5\") " pod="openshift-dns/dns-default-cw4qm"
Jul 22 21:01:43.071368 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:43.071307    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6fmcx\""
Jul 22 21:01:43.080064 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:43.080044    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cw4qm"
Jul 22 21:01:43.201738 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:43.201711    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cw4qm"]
Jul 22 21:01:43.204534 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:43.204511    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d8d735_5141_43ff_a8b9_6a35615d62b5.slice/crio-1e5e9b344c59577acc1a7fa7acefbd47142b9125b8f16c447a83e8ab435911d6 WatchSource:0}: Error finding container 1e5e9b344c59577acc1a7fa7acefbd47142b9125b8f16c447a83e8ab435911d6: Status 404 returned error can't find the container with id 1e5e9b344c59577acc1a7fa7acefbd47142b9125b8f16c447a83e8ab435911d6
Jul 22 21:01:44.084118 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:44.084080    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cw4qm" event={"ID":"10d8d735-5141-43ff-a8b9-6a35615d62b5","Type":"ContainerStarted","Data":"1e5e9b344c59577acc1a7fa7acefbd47142b9125b8f16c447a83e8ab435911d6"}
Jul 22 21:01:45.088884 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:45.088813    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cw4qm" event={"ID":"10d8d735-5141-43ff-a8b9-6a35615d62b5","Type":"ContainerStarted","Data":"959f7dfab878358783bf58f12b1add9f6a257a7ee43bf2a462e94e9938ee84b1"}
Jul 22 21:01:45.088884 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:45.088862    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cw4qm" event={"ID":"10d8d735-5141-43ff-a8b9-6a35615d62b5","Type":"ContainerStarted","Data":"1c44d193d2acdeb26a39be53dfb9d253d06f80d49a2b32ca7c6ac30e1ee14a14"}
Jul 22 21:01:45.089276 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:45.088891    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cw4qm"
Jul 22 21:01:45.105593 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:45.105551    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cw4qm" podStartSLOduration=251.583610526 podStartE2EDuration="4m13.10553705s" totalImagesPullingTime="1.521926524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:01:45.103864459 +0000 UTC m=+287.504666452" watchObservedRunningTime="2026-07-22 21:01:45.10553705 +0000 UTC m=+287.506339032"
Jul 22 21:01:45.542684 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:45.542654    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:46.785796 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:46.785754    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:47.043922 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:47.043841    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:47.144487 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:47.144453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:48.414894 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:48.414851    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:48.554653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:48.554626    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:48.732917 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:48.732852    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:51.019115 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:51.019081    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:51.073406 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:51.073379    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b62h4" podUID="4df97b04-40ba-466d-a735-cd1184db9185"
Jul 22 21:01:51.138070 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:51.138042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:51.214567 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:51.214543    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:51.381104 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:51.381038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:51.721220 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:51.721172    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:51.822980 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:51.822949    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:51.927804 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:51.927765    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:52.029805 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:52.029717    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:52.324829 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:52.324755    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:52.424277 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:52.424251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:52.945632 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:52.945602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:53.161003 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.160976    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:53.507290 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.507261    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:53.682189 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.682140    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:53.899368 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.899338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:53.958591 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.958562    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6wwc8"]
Jul 22 21:01:53.961143 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.961128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:53.963631 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.963614    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8pdzx\""
Jul 22 21:01:53.963754 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.963664    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 21:01:53.965309 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.965268    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 21:01:53.965412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.965388    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 21:01:53.965485 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.965471    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 21:01:53.965705 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.965687    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 21:01:53.965914 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:53.965896    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 21:01:54.005206 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.005165    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:54.060500 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-root\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060595 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060595 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbn8\" (UniqueName: \"kubernetes.io/projected/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-kube-api-access-msbn8\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060673 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-sys\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060673 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-metrics-client-ca\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060748 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-textfile\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060748 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060698    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-wtmp\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060748 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.060856 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.060770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-tls\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.079522 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.079497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:54.161275 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-root\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161275 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161275 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbn8\" (UniqueName: \"kubernetes.io/projected/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-kube-api-access-msbn8\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-sys\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-metrics-client-ca\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161347    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-textfile\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-sys\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-wtmp\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-root\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161440    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-tls\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:54.161528    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161533    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-wtmp\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.161709 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:01:54.161606    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-tls podName:a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777 nodeName:}" failed. No retries permitted until 2026-07-22 21:01:54.661586525 +0000 UTC m=+297.062388490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-tls") pod "node-exporter-6wwc8" (UID: "a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777") : secret "node-exporter-tls" not found
Jul 22 21:01:54.162327 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161716    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-textfile\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.162327 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.161903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-metrics-client-ca\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.162412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.162333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.163536 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.163519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.170788 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.170769    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbn8\" (UniqueName: \"kubernetes.io/projected/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-kube-api-access-msbn8\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.244518 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.244469    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:54.343626 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.343598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:54.443609 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.443538    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:54.607531 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.607503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:54.664523 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.664500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-tls\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.666666 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.666646    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777-node-exporter-tls\") pod \"node-exporter-6wwc8\" (UID: \"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777\") " pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.805458 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.805372    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:54.870261 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.870237    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wwc8"
Jul 22 21:01:54.880689 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:01:54.880663    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29e2c2a_53a6_4fe4_8f1d_b92dbaca2777.slice/crio-d536ec5cc1a0f2ccb978962e461843fbfb6f19351448fcfe7e8467d37bf8d99b WatchSource:0}: Error finding container d536ec5cc1a0f2ccb978962e461843fbfb6f19351448fcfe7e8467d37bf8d99b: Status 404 returned error can't find the container with id d536ec5cc1a0f2ccb978962e461843fbfb6f19351448fcfe7e8467d37bf8d99b
Jul 22 21:01:54.923450 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:54.923415    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:55.024376 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:55.024350    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:55.094017 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:55.093957    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cw4qm"
Jul 22 21:01:55.119106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:55.119082    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwc8" event={"ID":"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777","Type":"ContainerStarted","Data":"d536ec5cc1a0f2ccb978962e461843fbfb6f19351448fcfe7e8467d37bf8d99b"}
Jul 22 21:01:55.133340 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:55.133309    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:55.398314 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:55.398278    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:56.056218 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:56.056173    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-856988977b-txn6q"
Jul 22 21:01:56.165807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:56.165774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:56.256075 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:56.256043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:56.356071 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:56.356002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:56.633231 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:56.633136    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:56.735781 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:56.735752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:56.925343 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:56.925312    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:57.023521 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:57.023489    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:57.139520 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:57.139493    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:57.238522 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:57.238455    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:57.939123 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:57.939098    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:01:57.941935 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:57.941911    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:01:57.945379 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:57.945343    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 21:01:58.701114 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:58.701085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:58.848005 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:58.847936    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:59.134112 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:59.134035    2559 generic.go:356] "Generic (PLEG): container finished" podID="a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777" containerID="0ac9886ac12b450890b5f79e1aa6762c3f92f11d5e3cbfe09c8caf0b497e9e41" exitCode=0
Jul 22 21:01:59.134112 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:59.134099    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwc8" event={"ID":"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777","Type":"ContainerDied","Data":"0ac9886ac12b450890b5f79e1aa6762c3f92f11d5e3cbfe09c8caf0b497e9e41"}
Jul 22 21:01:59.821363 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:59.821329    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:01:59.919380 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:01:59.919349    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:00.020758 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:00.020724    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:00.139315 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:00.139288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwc8" event={"ID":"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777","Type":"ContainerStarted","Data":"d915cb8f298e9101020ae407378eb6670a386d7bc7e200459d6e1e7395555ce7"}
Jul 22 21:02:00.139684 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:00.139334    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wwc8" event={"ID":"a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777","Type":"ContainerStarted","Data":"71f477925d4f010e544aa8a9dc00e48b8320488fd2e89ed6796eae39bb04a324"}
Jul 22 21:02:00.161875 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:00.161835    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6wwc8" podStartSLOduration=3.513344957 podStartE2EDuration="7.161822311s" totalImagesPullingTime="3.586900321s" totalInitContainerRuntime="61.577033ms" isStatefulPod=true podCreationTimestamp="2026-07-22 21:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:02:00.160400054 +0000 UTC m=+302.561202068" watchObservedRunningTime="2026-07-22 21:02:00.161822311 +0000 UTC m=+302.562624323"
Jul 22 21:02:00.211622 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:00.211590    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:05.377430 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:05.377396    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:05.482544 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:05.482515    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:06.071021 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:06.070995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 21:02:08.186002 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:08.185966    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:08.256806 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:08.256774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:08.912447 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:08.912414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:09.016481 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.016453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:09.083545 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.083519    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bb986974-wdbst"]
Jul 22 21:02:09.111064 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.111046    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb986974-wdbst"]
Jul 22 21:02:09.111171 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.111125    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.113505 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.113486    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 22 21:02:09.113617 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.113595    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 22 21:02:09.113680 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.113614    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 22 21:02:09.113680 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.113620    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 22 21:02:09.113785 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.113689    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 22 21:02:09.114272 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.114247    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 22 21:02:09.114364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.114319    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 22 21:02:09.114364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.114355    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-256jf\""
Jul 22 21:02:09.118258 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.118242    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 22 21:02:09.160652 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.160632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkb8p\" (UniqueName: \"kubernetes.io/projected/e7b36625-0df9-4cf8-ba0b-375a2687764b-kube-api-access-qkb8p\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.160755 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.160659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-oauth-config\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.160755 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.160676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-config\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.160755 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.160693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-trusted-ca-bundle\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.160755 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.160708    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-oauth-serving-cert\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.160907 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.160812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-service-ca\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.160907 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.160848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-serving-cert\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.261666 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.261611    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-serving-cert\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.261666 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.261648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkb8p\" (UniqueName: \"kubernetes.io/projected/e7b36625-0df9-4cf8-ba0b-375a2687764b-kube-api-access-qkb8p\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.261666 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.261665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-oauth-config\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262085 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.261681    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-config\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262085 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.261703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-trusted-ca-bundle\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262085 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.261739    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-oauth-serving-cert\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262085 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.261806    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-service-ca\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.262465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-config\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262565 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.262487    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-service-ca\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262565 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.262536    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-oauth-serving-cert\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.262751 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.262730    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-trusted-ca-bundle\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.264154 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.264135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-oauth-config\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.264248 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.264234    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-serving-cert\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.270435 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.270407    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkb8p\" (UniqueName: \"kubernetes.io/projected/e7b36625-0df9-4cf8-ba0b-375a2687764b-kube-api-access-qkb8p\") pod \"console-6bb986974-wdbst\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") " pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.420893 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.420867    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:09.541436 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.541411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb986974-wdbst"]
Jul 22 21:02:09.543820 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:02:09.543792    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b36625_0df9_4cf8_ba0b_375a2687764b.slice/crio-8d76343b9c8501571d134972a8bfecac8ec27fd6bf923dad5a34932a375efee6 WatchSource:0}: Error finding container 8d76343b9c8501571d134972a8bfecac8ec27fd6bf923dad5a34932a375efee6: Status 404 returned error can't find the container with id 8d76343b9c8501571d134972a8bfecac8ec27fd6bf923dad5a34932a375efee6
Jul 22 21:02:09.545780 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:09.545763    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 21:02:10.174688 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:10.174655    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb986974-wdbst" event={"ID":"e7b36625-0df9-4cf8-ba0b-375a2687764b","Type":"ContainerStarted","Data":"8d76343b9c8501571d134972a8bfecac8ec27fd6bf923dad5a34932a375efee6"}
Jul 22 21:02:15.192686 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:15.192614    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb986974-wdbst" event={"ID":"e7b36625-0df9-4cf8-ba0b-375a2687764b","Type":"ContainerStarted","Data":"788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224"}
Jul 22 21:02:15.214111 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:15.214065    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bb986974-wdbst" podStartSLOduration=0.830894307 podStartE2EDuration="6.214051764s" totalImagesPullingTime="5.383157457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:02:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:02:15.212528709 +0000 UTC m=+317.613330689" watchObservedRunningTime="2026-07-22 21:02:15.214051764 +0000 UTC m=+317.614853747"
Jul 22 21:02:16.388704 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:16.388667    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:16.468668 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:16.468643    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:18.919089 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:18.919057    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:18.998484 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:18.998457    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:19.421727 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:19.421705    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:19.421860 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:19.421751    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:19.426443 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:19.426423    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:20.119018 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:20.118981    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:20.211993 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:20.211966    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:02:20.218626 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:20.218605    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:20.357778 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:20.357747    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:20.410287 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:20.410258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:21.038632 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:21.038598    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:21.172360 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:21.172322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:21.268979 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:21.268951    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:22.656096 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:22.656065    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mqn7j_ac9cb0ea-7a64-428e-b599-b13b45e763ce/serve-healthcheck-canary/0.log"
Jul 22 21:02:24.334636 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:24.334600    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:24.416043 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:24.416010    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:34.256746 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:34.256677    2559 generic.go:356] "Generic (PLEG): container finished" podID="0c0c8c06-e310-400c-8a0b-be33755f5f79" containerID="4d2106c943afa820a01ee10102d3642e1b78af187dcdb13a87f8cc65c41b244f" exitCode=0
Jul 22 21:02:34.257073 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:34.256745    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467" event={"ID":"0c0c8c06-e310-400c-8a0b-be33755f5f79","Type":"ContainerDied","Data":"4d2106c943afa820a01ee10102d3642e1b78af187dcdb13a87f8cc65c41b244f"}
Jul 22 21:02:34.257073 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:34.257061    2559 scope.go:117] "RemoveContainer" containerID="4d2106c943afa820a01ee10102d3642e1b78af187dcdb13a87f8cc65c41b244f"
Jul 22 21:02:35.262936 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:35.262903    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-hq467" event={"ID":"0c0c8c06-e310-400c-8a0b-be33755f5f79","Type":"ContainerStarted","Data":"119d2e85f97c2fbee3e151b6577643a033b94c977e7ed7fc25ab005918ee63ee"}
Jul 22 21:02:39.311736 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:39.311697    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:42.253654 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:42.253615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:45.401409 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:45.401374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:45.512922 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:45.512886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:45.655462 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:45.655394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:58.800398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:58.800305    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:02:58.902027 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:02:58.902000    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:10.077764 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:10.077729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:10.182518 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:10.182489    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:12.091729 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:12.091702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 21:03:12.093992 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:12.093969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4df97b04-40ba-466d-a735-cd1184db9185-metrics-certs\") pod \"network-metrics-daemon-b62h4\" (UID: \"4df97b04-40ba-466d-a735-cd1184db9185\") " pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 21:03:12.374465 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:12.374409    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j26qm\""
Jul 22 21:03:12.382414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:12.382396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b62h4"
Jul 22 21:03:12.526927 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:12.526897    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b62h4"]
Jul 22 21:03:12.530011 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:03:12.529985    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df97b04_40ba_466d_a735_cd1184db9185.slice/crio-62fec39932f1b73e1db12487738692faf2411d6c99eb4f6769565ace1319e92e WatchSource:0}: Error finding container 62fec39932f1b73e1db12487738692faf2411d6c99eb4f6769565ace1319e92e: Status 404 returned error can't find the container with id 62fec39932f1b73e1db12487738692faf2411d6c99eb4f6769565ace1319e92e
Jul 22 21:03:12.531106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:12.531072    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:12.649802 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:12.649762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:13.384191 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:13.384135    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b62h4" event={"ID":"4df97b04-40ba-466d-a735-cd1184db9185","Type":"ContainerStarted","Data":"62fec39932f1b73e1db12487738692faf2411d6c99eb4f6769565ace1319e92e"}
Jul 22 21:03:14.388654 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:14.388621    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b62h4" event={"ID":"4df97b04-40ba-466d-a735-cd1184db9185","Type":"ContainerStarted","Data":"bec5e4715233a7cfc0e455ac9929e49909b49c90d2c59a8129519f0987e3bae5"}
Jul 22 21:03:14.388998 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:14.388670    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b62h4" event={"ID":"4df97b04-40ba-466d-a735-cd1184db9185","Type":"ContainerStarted","Data":"3e399d5d8c4bf430b7c9414863c56db2432b54bd27cabe1b94fc938253899318"}
Jul 22 21:03:14.423139 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:14.423097    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b62h4" podStartSLOduration=375.50463008 podStartE2EDuration="6m16.423082698s" totalImagesPullingTime="918.452618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 20:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:03:14.422292604 +0000 UTC m=+376.823094608" watchObservedRunningTime="2026-07-22 21:03:14.423082698 +0000 UTC m=+376.823884680"
Jul 22 21:03:14.567668 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:14.567641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:14.647230 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:14.647199    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:20.576430 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:20.576392    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:20.648526 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:20.648500    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:21.982728 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:21.982700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:22.054488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:22.054462    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:31.327801 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:31.327768    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bb986974-wdbst"]
Jul 22 21:03:31.445137 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:31.445088    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:31.551653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:31.551414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:31.620163 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:31.620077    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:39.318636 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:39.318603    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:39.419655 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:39.419620    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:39.519189 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:39.519153    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:52.875146 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:52.875106    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:52.975422 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:52.975369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:03:56.466057 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.466022    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bb986974-wdbst" podUID="e7b36625-0df9-4cf8-ba0b-375a2687764b" containerName="console" containerID="cri-o://788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224" gracePeriod=15
Jul 22 21:03:56.700798 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.700776    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:03:56.791153 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791098    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-serving-cert\") pod \"e7b36625-0df9-4cf8-ba0b-375a2687764b\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") "
Jul 22 21:03:56.791153 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-trusted-ca-bundle\") pod \"e7b36625-0df9-4cf8-ba0b-375a2687764b\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") "
Jul 22 21:03:56.791153 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791151    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-oauth-config\") pod \"e7b36625-0df9-4cf8-ba0b-375a2687764b\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") "
Jul 22 21:03:56.791385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791218    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-oauth-serving-cert\") pod \"e7b36625-0df9-4cf8-ba0b-375a2687764b\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") "
Jul 22 21:03:56.791385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791246    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-config\") pod \"e7b36625-0df9-4cf8-ba0b-375a2687764b\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") "
Jul 22 21:03:56.791385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791289    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkb8p\" (UniqueName: \"kubernetes.io/projected/e7b36625-0df9-4cf8-ba0b-375a2687764b-kube-api-access-qkb8p\") pod \"e7b36625-0df9-4cf8-ba0b-375a2687764b\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") "
Jul 22 21:03:56.791385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791320    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-service-ca\") pod \"e7b36625-0df9-4cf8-ba0b-375a2687764b\" (UID: \"e7b36625-0df9-4cf8-ba0b-375a2687764b\") "
Jul 22 21:03:56.791695 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791666    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e7b36625-0df9-4cf8-ba0b-375a2687764b" (UID: "e7b36625-0df9-4cf8-ba0b-375a2687764b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:03:56.791812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791708    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e7b36625-0df9-4cf8-ba0b-375a2687764b" (UID: "e7b36625-0df9-4cf8-ba0b-375a2687764b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:03:56.791812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791738    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-config" (OuterVolumeSpecName: "console-config") pod "e7b36625-0df9-4cf8-ba0b-375a2687764b" (UID: "e7b36625-0df9-4cf8-ba0b-375a2687764b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:03:56.791812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.791809    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-service-ca" (OuterVolumeSpecName: "service-ca") pod "e7b36625-0df9-4cf8-ba0b-375a2687764b" (UID: "e7b36625-0df9-4cf8-ba0b-375a2687764b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:03:56.793287 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.793265    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e7b36625-0df9-4cf8-ba0b-375a2687764b" (UID: "e7b36625-0df9-4cf8-ba0b-375a2687764b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:03:56.793707 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.793692    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e7b36625-0df9-4cf8-ba0b-375a2687764b" (UID: "e7b36625-0df9-4cf8-ba0b-375a2687764b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:03:56.793758 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.793720    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7b36625-0df9-4cf8-ba0b-375a2687764b-kube-api-access-qkb8p" (OuterVolumeSpecName: "kube-api-access-qkb8p") pod "e7b36625-0df9-4cf8-ba0b-375a2687764b" (UID: "e7b36625-0df9-4cf8-ba0b-375a2687764b"). InnerVolumeSpecName "kube-api-access-qkb8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:03:56.892282 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.892262    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-oauth-serving-cert\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:03:56.892282 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.892281    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-config\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:03:56.892412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.892292    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkb8p\" (UniqueName: \"kubernetes.io/projected/e7b36625-0df9-4cf8-ba0b-375a2687764b-kube-api-access-qkb8p\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:03:56.892412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.892300    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:03:56.892412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.892309    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-serving-cert\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:03:56.892412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.892316    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b36625-0df9-4cf8-ba0b-375a2687764b-trusted-ca-bundle\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:03:56.892412 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:56.892324    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e7b36625-0df9-4cf8-ba0b-375a2687764b-console-oauth-config\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:03:57.534662 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.534628    2559 generic.go:356] "Generic (PLEG): container finished" podID="e7b36625-0df9-4cf8-ba0b-375a2687764b" containerID="788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224" exitCode=0
Jul 22 21:03:57.535110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.534689    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb986974-wdbst" event={"ID":"e7b36625-0df9-4cf8-ba0b-375a2687764b","Type":"ContainerDied","Data":"788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224"}
Jul 22 21:03:57.535110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.534731    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb986974-wdbst" event={"ID":"e7b36625-0df9-4cf8-ba0b-375a2687764b","Type":"ContainerDied","Data":"8d76343b9c8501571d134972a8bfecac8ec27fd6bf923dad5a34932a375efee6"}
Jul 22 21:03:57.535110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.534757    2559 scope.go:117] "RemoveContainer" containerID="788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224"
Jul 22 21:03:57.535110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.534912    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb986974-wdbst"
Jul 22 21:03:57.549531 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.549510    2559 scope.go:117] "RemoveContainer" containerID="788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224"
Jul 22 21:03:57.549772 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:03:57.549751    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224\": container with ID starting with 788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224 not found: ID does not exist" containerID="788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224"
Jul 22 21:03:57.549818 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.549780    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224"} err="failed to get container status \"788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224\": rpc error: code = NotFound desc = could not find container \"788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224\": container with ID starting with 788a92b2e4fac7fded2b50c9881e8de42cae7234e58367162600110d782ab224 not found: ID does not exist"
Jul 22 21:03:57.561307 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.561284    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bb986974-wdbst"]
Jul 22 21:03:57.565871 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:57.565854    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bb986974-wdbst"]
Jul 22 21:03:58.076066 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:03:58.076035    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7b36625-0df9-4cf8-ba0b-375a2687764b" path="/var/lib/kubelet/pods/e7b36625-0df9-4cf8-ba0b-375a2687764b/volumes"
Jul 22 21:04:19.367187 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:19.367150    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:04:19.511754 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:19.511728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:04:49.178970 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:49.178936    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:04:57.687133 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:57.687097    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:04:57.787394 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:57.787366    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:04:58.943947 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:58.943880    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:04:59.630706 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.630675    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"]
Jul 22 21:04:59.630944 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.630930    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7b36625-0df9-4cf8-ba0b-375a2687764b" containerName="console"
Jul 22 21:04:59.630994 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.630947    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7b36625-0df9-4cf8-ba0b-375a2687764b" containerName="console"
Jul 22 21:04:59.631028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.631002    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7b36625-0df9-4cf8-ba0b-375a2687764b" containerName="console"
Jul 22 21:04:59.633707 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.633692    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:04:59.636527 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.636505    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 22 21:04:59.636659 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.636518    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 22 21:04:59.636659 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.636595    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xr7x6\""
Jul 22 21:04:59.642402 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.642382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"]
Jul 22 21:04:59.791052 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.791025    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttzcq\" (UniqueName: \"kubernetes.io/projected/ce5e8f5b-314a-48a9-8b25-7d101cff4f2c-kube-api-access-ttzcq\") pod \"cert-manager-cainjector-559cdd8c87-h5rqb\" (UID: \"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:04:59.791172 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.791061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce5e8f5b-314a-48a9-8b25-7d101cff4f2c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h5rqb\" (UID: \"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:04:59.892324 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.892301    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttzcq\" (UniqueName: \"kubernetes.io/projected/ce5e8f5b-314a-48a9-8b25-7d101cff4f2c-kube-api-access-ttzcq\") pod \"cert-manager-cainjector-559cdd8c87-h5rqb\" (UID: \"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:04:59.892398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.892333    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce5e8f5b-314a-48a9-8b25-7d101cff4f2c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h5rqb\" (UID: \"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:04:59.901348 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.901327    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce5e8f5b-314a-48a9-8b25-7d101cff4f2c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h5rqb\" (UID: \"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:04:59.901430 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.901398    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttzcq\" (UniqueName: \"kubernetes.io/projected/ce5e8f5b-314a-48a9-8b25-7d101cff4f2c-kube-api-access-ttzcq\") pod \"cert-manager-cainjector-559cdd8c87-h5rqb\" (UID: \"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:04:59.942446 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:04:59.942428    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"
Jul 22 21:05:00.061443 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:00.061404    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb"]
Jul 22 21:05:00.065052 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:05:00.065024    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce5e8f5b_314a_48a9_8b25_7d101cff4f2c.slice/crio-74e9f254edf1b4f1ea9ed49a4ca6145c816ecc1a93bb8a71d8d807b345a72e15 WatchSource:0}: Error finding container 74e9f254edf1b4f1ea9ed49a4ca6145c816ecc1a93bb8a71d8d807b345a72e15: Status 404 returned error can't find the container with id 74e9f254edf1b4f1ea9ed49a4ca6145c816ecc1a93bb8a71d8d807b345a72e15
Jul 22 21:05:00.745702 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:00.745675    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb" event={"ID":"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c","Type":"ContainerStarted","Data":"74e9f254edf1b4f1ea9ed49a4ca6145c816ecc1a93bb8a71d8d807b345a72e15"}
Jul 22 21:05:01.207807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:01.207763    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:03.288051 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:03.288024    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:03.759523 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:03.759488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb" event={"ID":"ce5e8f5b-314a-48a9-8b25-7d101cff4f2c","Type":"ContainerStarted","Data":"6167d6e7858569e23d522108b290c5db6f86a0026492519d68a3d9c73350ecf1"}
Jul 22 21:05:03.794233 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:03.794205    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:03.854773 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:03.854730    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h5rqb" podStartSLOduration=1.745256115 podStartE2EDuration="4.854718s" totalImagesPullingTime="3.109461885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:05:03.845535935 +0000 UTC m=+486.246337936" watchObservedRunningTime="2026-07-22 21:05:03.854718 +0000 UTC m=+486.255519981"
Jul 22 21:05:07.151055 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.151013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:07.532903 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.532830    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-l9w9q"]
Jul 22 21:05:07.536056 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.536039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:07.538228 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.538210    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-d7bbg\""
Jul 22 21:05:07.547492 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.547471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-l9w9q"]
Jul 22 21:05:07.655563 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.655543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4-bound-sa-token\") pod \"cert-manager-55b4d954d5-l9w9q\" (UID: \"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4\") " pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:07.655694 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.655571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kph8\" (UniqueName: \"kubernetes.io/projected/c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4-kube-api-access-5kph8\") pod \"cert-manager-55b4d954d5-l9w9q\" (UID: \"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4\") " pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:07.756667 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.756638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4-bound-sa-token\") pod \"cert-manager-55b4d954d5-l9w9q\" (UID: \"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4\") " pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:07.756789 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.756674    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kph8\" (UniqueName: \"kubernetes.io/projected/c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4-kube-api-access-5kph8\") pod \"cert-manager-55b4d954d5-l9w9q\" (UID: \"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4\") " pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:07.766941 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.766913    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4-bound-sa-token\") pod \"cert-manager-55b4d954d5-l9w9q\" (UID: \"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4\") " pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:07.767121 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.767103    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kph8\" (UniqueName: \"kubernetes.io/projected/c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4-kube-api-access-5kph8\") pod \"cert-manager-55b4d954d5-l9w9q\" (UID: \"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4\") " pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:07.844040 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:07.843981    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-l9w9q"
Jul 22 21:05:08.001761 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:08.001727    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-l9w9q"]
Jul 22 21:05:08.004555 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:05:08.004533    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc60d9ba7_50bf_4bd7_85e7_75f97db3d3f4.slice/crio-44e0f3a3a3819102c01dcda2eb4bd74004de66135b308e0c9dd255155925da4e WatchSource:0}: Error finding container 44e0f3a3a3819102c01dcda2eb4bd74004de66135b308e0c9dd255155925da4e: Status 404 returned error can't find the container with id 44e0f3a3a3819102c01dcda2eb4bd74004de66135b308e0c9dd255155925da4e
Jul 22 21:05:08.778954 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:08.778919    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-l9w9q" event={"ID":"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4","Type":"ContainerStarted","Data":"c781a28bb3a250501cc7ff57b685c41ccd45d6de0a33f074f1a6519fda05c26b"}
Jul 22 21:05:08.778954 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:08.778960    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-l9w9q" event={"ID":"c60d9ba7-50bf-4bd7-85e7-75f97db3d3f4","Type":"ContainerStarted","Data":"44e0f3a3a3819102c01dcda2eb4bd74004de66135b308e0c9dd255155925da4e"}
Jul 22 21:05:08.799136 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:08.799090    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-l9w9q" podStartSLOduration=1.799076729 podStartE2EDuration="1.799076729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:05:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:05:08.795851593 +0000 UTC m=+491.196653575" watchObservedRunningTime="2026-07-22 21:05:08.799076729 +0000 UTC m=+491.199878712"
Jul 22 21:05:14.296399 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:14.296367    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:14.399153 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:14.399130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:21.540868 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:21.540826    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:21.679280 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:21.679236    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:23.567095 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:23.567059    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:24.193403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:24.193366    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:24.295066 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:24.295025    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:24.732902 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:24.732870    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:24.811737 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:24.811703    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:36.222298 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:36.222260    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:36.318108 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:36.318079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:36.869481 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:36.869438    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:37.170414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.170358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:37.276866 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.276819    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:37.314156 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.314129    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"]
Jul 22 21:05:37.353491 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.353471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"]
Jul 22 21:05:37.353623 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.353559    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.357111 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.357093    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 21:05:37.357613 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.357588    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 21:05:37.357700 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.357588    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wglmk\""
Jul 22 21:05:37.360308 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.360287    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 21:05:37.360643 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.360623    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 21:05:37.545044 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.544990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c03d398d-e157-45a0-892d-cca57f13c587-tmp\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.545044 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.545028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c03d398d-e157-45a0-892d-cca57f13c587-apiservice-cert\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.545231 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.545053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c03d398d-e157-45a0-892d-cca57f13c587-manifests\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.545231 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.545138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqrxs\" (UniqueName: \"kubernetes.io/projected/c03d398d-e157-45a0-892d-cca57f13c587-kube-api-access-nqrxs\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.545330 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.545247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c03d398d-e157-45a0-892d-cca57f13c587-webhook-cert\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.646294 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.646267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c03d398d-e157-45a0-892d-cca57f13c587-webhook-cert\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.646403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.646314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c03d398d-e157-45a0-892d-cca57f13c587-tmp\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.646403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.646351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c03d398d-e157-45a0-892d-cca57f13c587-apiservice-cert\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.646403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.646379    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c03d398d-e157-45a0-892d-cca57f13c587-manifests\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.646566 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.646406    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqrxs\" (UniqueName: \"kubernetes.io/projected/c03d398d-e157-45a0-892d-cca57f13c587-kube-api-access-nqrxs\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.646816 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.646791    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c03d398d-e157-45a0-892d-cca57f13c587-manifests\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.648833 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.648811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c03d398d-e157-45a0-892d-cca57f13c587-webhook-cert\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.649011 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.648988    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c03d398d-e157-45a0-892d-cca57f13c587-tmp\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.649160 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.649142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c03d398d-e157-45a0-892d-cca57f13c587-apiservice-cert\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.679376 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.679342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:37.685609 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.685586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqrxs\" (UniqueName: \"kubernetes.io/projected/c03d398d-e157-45a0-892d-cca57f13c587-kube-api-access-nqrxs\") pod \"opendatahub-operator-controller-manager-846644596-wddxk\" (UID: \"c03d398d-e157-45a0-892d-cca57f13c587\") " pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.772280 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.772242    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:37.890241 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.890219    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:37.964643 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.964614    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:37.975819 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:37.975796    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:38.078385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:38.078358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:38.107780 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:38.107681    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"]
Jul 22 21:05:38.110672 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:05:38.110645    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03d398d_e157_45a0_892d_cca57f13c587.slice/crio-83c9dc6f11891c779355c67ab74f6b9df9acdcc3da0270ca697961192f29c222 WatchSource:0}: Error finding container 83c9dc6f11891c779355c67ab74f6b9df9acdcc3da0270ca697961192f29c222: Status 404 returned error can't find the container with id 83c9dc6f11891c779355c67ab74f6b9df9acdcc3da0270ca697961192f29c222
Jul 22 21:05:38.181092 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:38.181031    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:38.274421 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:38.274395    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:38.878673 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:38.878634    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk" event={"ID":"c03d398d-e157-45a0-892d-cca57f13c587","Type":"ContainerStarted","Data":"83c9dc6f11891c779355c67ab74f6b9df9acdcc3da0270ca697961192f29c222"}
Jul 22 21:05:39.850919 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:39.850852    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:39.951814 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:39.951779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:41.893906 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:41.893872    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk" event={"ID":"c03d398d-e157-45a0-892d-cca57f13c587","Type":"ContainerStarted","Data":"e1efb237340e53bc11dc523529842270ca2b79b7da657a170b9be38840cde888"}
Jul 22 21:05:42.898843 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:42.898811    2559 generic.go:356] "Generic (PLEG): container finished" podID="c03d398d-e157-45a0-892d-cca57f13c587" containerID="e1efb237340e53bc11dc523529842270ca2b79b7da657a170b9be38840cde888" exitCode=0
Jul 22 21:05:42.899261 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:42.898901    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk" event={"ID":"c03d398d-e157-45a0-892d-cca57f13c587","Type":"ContainerDied","Data":"e1efb237340e53bc11dc523529842270ca2b79b7da657a170b9be38840cde888"}
Jul 22 21:05:43.904888 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:43.904858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk" event={"ID":"c03d398d-e157-45a0-892d-cca57f13c587","Type":"ContainerStarted","Data":"fa916579e39bc30a6eca724ad8fa9c4dda69e3ad88d236265dfba965933f2618"}
Jul 22 21:05:43.905307 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:43.904974    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:43.930667 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:43.930617    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk" podStartSLOduration=2.707087286 podStartE2EDuration="6.930605191s" totalImagesPullingTime="3.190114305s" totalInitContainerRuntime="1.0334036s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:05:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:05:43.930462083 +0000 UTC m=+526.331264098" watchObservedRunningTime="2026-07-22 21:05:43.930605191 +0000 UTC m=+526.331407173"
Jul 22 21:05:48.823347 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:48.823307    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:48.918693 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:48.918659    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:49.080097 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:49.080027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:49.176964 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:49.176936    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:05:54.911087 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:54.911014    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-846644596-wddxk"
Jul 22 21:05:59.928231 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:05:59.928172    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:00.086859 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.086815    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:00.756780 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.756748    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fzq8x"]
Jul 22 21:06:00.759876 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.759857    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:00.762869 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.762848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 22 21:06:00.762869 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.762864    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xptrh\""
Jul 22 21:06:00.774459 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.774440    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fzq8x"]
Jul 22 21:06:00.803969 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.803943    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jl9x\" (UniqueName: \"kubernetes.io/projected/786ef4eb-1d55-4cdc-af35-895719287f5e-kube-api-access-4jl9x\") pod \"odh-model-controller-7998c67dc4-fzq8x\" (UID: \"786ef4eb-1d55-4cdc-af35-895719287f5e\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:00.804077 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.803989    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/786ef4eb-1d55-4cdc-af35-895719287f5e-cert\") pod \"odh-model-controller-7998c67dc4-fzq8x\" (UID: \"786ef4eb-1d55-4cdc-af35-895719287f5e\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:00.821740 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.821692    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:00.905267 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.905230    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jl9x\" (UniqueName: \"kubernetes.io/projected/786ef4eb-1d55-4cdc-af35-895719287f5e-kube-api-access-4jl9x\") pod \"odh-model-controller-7998c67dc4-fzq8x\" (UID: \"786ef4eb-1d55-4cdc-af35-895719287f5e\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:00.905356 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.905272    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/786ef4eb-1d55-4cdc-af35-895719287f5e-cert\") pod \"odh-model-controller-7998c67dc4-fzq8x\" (UID: \"786ef4eb-1d55-4cdc-af35-895719287f5e\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:00.905397 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:00.905375    2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 22 21:06:00.905434 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:00.905423    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/786ef4eb-1d55-4cdc-af35-895719287f5e-cert podName:786ef4eb-1d55-4cdc-af35-895719287f5e nodeName:}" failed. No retries permitted until 2026-07-22 21:06:01.405407797 +0000 UTC m=+543.806209758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/786ef4eb-1d55-4cdc-af35-895719287f5e-cert") pod "odh-model-controller-7998c67dc4-fzq8x" (UID: "786ef4eb-1d55-4cdc-af35-895719287f5e") : secret "odh-model-controller-webhook-cert" not found
Jul 22 21:06:00.919065 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.919045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jl9x\" (UniqueName: \"kubernetes.io/projected/786ef4eb-1d55-4cdc-af35-895719287f5e-kube-api-access-4jl9x\") pod \"odh-model-controller-7998c67dc4-fzq8x\" (UID: \"786ef4eb-1d55-4cdc-af35-895719287f5e\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:00.942704 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:00.942681    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:01.042864 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.042794    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:01.146270 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.146245    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:01.409704 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.409676    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/786ef4eb-1d55-4cdc-af35-895719287f5e-cert\") pod \"odh-model-controller-7998c67dc4-fzq8x\" (UID: \"786ef4eb-1d55-4cdc-af35-895719287f5e\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:01.411950 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.411925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/786ef4eb-1d55-4cdc-af35-895719287f5e-cert\") pod \"odh-model-controller-7998c67dc4-fzq8x\" (UID: \"786ef4eb-1d55-4cdc-af35-895719287f5e\") " pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:01.446620 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.446594    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lbs79"]
Jul 22 21:06:01.448871 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.448854    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:01.450889 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.450871    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 22 21:06:01.462401 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.462382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lbs79"]
Jul 22 21:06:01.475790 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.475769    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:01.510366 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.510347    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-tls-certs\") pod \"model-serving-api-54d89bd5fc-lbs79\" (UID: \"26b03ead-8ee3-4bbc-bc12-0124700f9a9d\") " pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:01.510454 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.510401    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdfcj\" (UniqueName: \"kubernetes.io/projected/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-kube-api-access-xdfcj\") pod \"model-serving-api-54d89bd5fc-lbs79\" (UID: \"26b03ead-8ee3-4bbc-bc12-0124700f9a9d\") " pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:01.611307 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.611282    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-tls-certs\") pod \"model-serving-api-54d89bd5fc-lbs79\" (UID: \"26b03ead-8ee3-4bbc-bc12-0124700f9a9d\") " pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:01.611391 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.611362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdfcj\" (UniqueName: \"kubernetes.io/projected/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-kube-api-access-xdfcj\") pod \"model-serving-api-54d89bd5fc-lbs79\" (UID: \"26b03ead-8ee3-4bbc-bc12-0124700f9a9d\") " pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:01.611450 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:01.611434    2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 22 21:06:01.611508 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:01.611498    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-tls-certs podName:26b03ead-8ee3-4bbc-bc12-0124700f9a9d nodeName:}" failed. No retries permitted until 2026-07-22 21:06:02.111480077 +0000 UTC m=+544.512282053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-tls-certs") pod "model-serving-api-54d89bd5fc-lbs79" (UID: "26b03ead-8ee3-4bbc-bc12-0124700f9a9d") : secret "model-serving-api-tls" not found
Jul 22 21:06:01.622558 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.622529    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdfcj\" (UniqueName: \"kubernetes.io/projected/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-kube-api-access-xdfcj\") pod \"model-serving-api-54d89bd5fc-lbs79\" (UID: \"26b03ead-8ee3-4bbc-bc12-0124700f9a9d\") " pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:01.669335 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.669275    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:01.806621 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.806586    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fzq8x"]
Jul 22 21:06:01.809443 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:06:01.809418    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786ef4eb_1d55_4cdc_af35_895719287f5e.slice/crio-4757c2dc8d3a5d6a3ede31e29a2f771db88b90f7bafef169f15b4e2d3f3a502f WatchSource:0}: Error finding container 4757c2dc8d3a5d6a3ede31e29a2f771db88b90f7bafef169f15b4e2d3f3a502f: Status 404 returned error can't find the container with id 4757c2dc8d3a5d6a3ede31e29a2f771db88b90f7bafef169f15b4e2d3f3a502f
Jul 22 21:06:01.890350 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.890321    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:01.969274 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:01.969223    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x" event={"ID":"786ef4eb-1d55-4cdc-af35-895719287f5e","Type":"ContainerStarted","Data":"4757c2dc8d3a5d6a3ede31e29a2f771db88b90f7bafef169f15b4e2d3f3a502f"}
Jul 22 21:06:02.116653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:02.116630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-tls-certs\") pod \"model-serving-api-54d89bd5fc-lbs79\" (UID: \"26b03ead-8ee3-4bbc-bc12-0124700f9a9d\") " pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:02.119279 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:02.119262    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26b03ead-8ee3-4bbc-bc12-0124700f9a9d-tls-certs\") pod \"model-serving-api-54d89bd5fc-lbs79\" (UID: \"26b03ead-8ee3-4bbc-bc12-0124700f9a9d\") " pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:02.358892 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:02.358835    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:02.486559 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:02.486537    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lbs79"]
Jul 22 21:06:02.488933 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:06:02.488905    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b03ead_8ee3_4bbc_bc12_0124700f9a9d.slice/crio-cb08779a503da98cd5f7e0c54527f3c3130bbbc0ae062eb9201632158b6eecaa WatchSource:0}: Error finding container cb08779a503da98cd5f7e0c54527f3c3130bbbc0ae062eb9201632158b6eecaa: Status 404 returned error can't find the container with id cb08779a503da98cd5f7e0c54527f3c3130bbbc0ae062eb9201632158b6eecaa
Jul 22 21:06:02.976123 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:02.976083    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lbs79" event={"ID":"26b03ead-8ee3-4bbc-bc12-0124700f9a9d","Type":"ContainerStarted","Data":"cb08779a503da98cd5f7e0c54527f3c3130bbbc0ae062eb9201632158b6eecaa"}
Jul 22 21:06:05.935476 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.935440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:05.988721 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.988674    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6766dd8-th24k"]
Jul 22 21:06:05.991410 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.991391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:05.992349 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.992317    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x" event={"ID":"786ef4eb-1d55-4cdc-af35-895719287f5e","Type":"ContainerStarted","Data":"1639eb818ef12f470b070fbc94955c466df7ae6489fd0ecd9b9118517a4eb1e7"}
Jul 22 21:06:05.992464 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.992390    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:05.993812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.993789    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lbs79" event={"ID":"26b03ead-8ee3-4bbc-bc12-0124700f9a9d","Type":"ContainerStarted","Data":"67ec4003a35bbcc2078b14a7696b723b2cd9645c89eeb880e958ac62b096084f"}
Jul 22 21:06:05.993935 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.993918    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:05.994036 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.994018    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 21:06:05.997216 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.997197    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jkqdq\""
Jul 22 21:06:05.997315 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.997242    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 21:06:05.997315 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.997252    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 21:06:05.997315 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:05.997307    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 21:06:06.004705 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.004679    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6766dd8-th24k"]
Jul 22 21:06:06.043910 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.043870    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x" podStartSLOduration=2.695012673 podStartE2EDuration="6.043857298s" totalImagesPullingTime="3.348844625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:06:06.04213728 +0000 UTC m=+548.442939273" watchObservedRunningTime="2026-07-22 21:06:06.043857298 +0000 UTC m=+548.444659280"
Jul 22 21:06:06.047807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.047789    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b18e2476-a2d3-4c30-bf0b-464aafab41f6-tls-certs\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.047875 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.047817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b18e2476-a2d3-4c30-bf0b-464aafab41f6-tmp\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.047912 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.047873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfd5\" (UniqueName: \"kubernetes.io/projected/b18e2476-a2d3-4c30-bf0b-464aafab41f6-kube-api-access-qcfd5\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.060146 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.060112    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lbs79" podStartSLOduration=2.395923479 podStartE2EDuration="5.060100411s" totalImagesPullingTime="2.664176932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:06:06.059942417 +0000 UTC m=+548.460744402" watchObservedRunningTime="2026-07-22 21:06:06.060100411 +0000 UTC m=+548.460902393"
Jul 22 21:06:06.149080 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.149056    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b18e2476-a2d3-4c30-bf0b-464aafab41f6-tls-certs\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.149193 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.149086    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b18e2476-a2d3-4c30-bf0b-464aafab41f6-tmp\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.149193 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.149115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfd5\" (UniqueName: \"kubernetes.io/projected/b18e2476-a2d3-4c30-bf0b-464aafab41f6-kube-api-access-qcfd5\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.151409 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.151386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b18e2476-a2d3-4c30-bf0b-464aafab41f6-tmp\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.151552 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.151536    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b18e2476-a2d3-4c30-bf0b-464aafab41f6-tls-certs\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.157493 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.157471    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfd5\" (UniqueName: \"kubernetes.io/projected/b18e2476-a2d3-4c30-bf0b-464aafab41f6-kube-api-access-qcfd5\") pod \"kube-auth-proxy-6766dd8-th24k\" (UID: \"b18e2476-a2d3-4c30-bf0b-464aafab41f6\") " pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.202887 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.202833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:06.301167 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.301148    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k"
Jul 22 21:06:06.426689 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.426659    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6766dd8-th24k"]
Jul 22 21:06:06.429867 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:06:06.429841    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18e2476_a2d3_4c30_bf0b_464aafab41f6.slice/crio-a9a57104c09718a4c98afd03f5e5ce4cd9a389b8044c5b2ce9d7454b1762cf99 WatchSource:0}: Error finding container a9a57104c09718a4c98afd03f5e5ce4cd9a389b8044c5b2ce9d7454b1762cf99: Status 404 returned error can't find the container with id a9a57104c09718a4c98afd03f5e5ce4cd9a389b8044c5b2ce9d7454b1762cf99
Jul 22 21:06:06.999122 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:06.999086    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k" event={"ID":"b18e2476-a2d3-4c30-bf0b-464aafab41f6","Type":"ContainerStarted","Data":"a9a57104c09718a4c98afd03f5e5ce4cd9a389b8044c5b2ce9d7454b1762cf99"}
Jul 22 21:06:07.578908 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.578846    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:07.596893 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.596863    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"]
Jul 22 21:06:07.599463 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.599440    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.601904 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:07.601876    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-tls\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" type="*v1.Secret"
Jul 22 21:06:07.602046 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:07.602009    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istio-kubeconfig\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" type="*v1.Secret"
Jul 22 21:06:07.602592 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:07.602561    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" podUID="f6d7a11e-a9a1-4103-97d4-685fc52f8434" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.612227 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:07.612197    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cacerts\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"cacerts\"" type="*v1.Secret"
Jul 22 21:06:07.612648 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:07.612621    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-openshift-gateway-dockercfg-98c6m\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-98c6m\"" type="*v1.Secret"
Jul 22 21:06:07.613042 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:07.613022    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap"
Jul 22 21:06:07.633680 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.633654    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"]
Jul 22 21:06:07.662115 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.662094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.662239 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.662127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.662239 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.662154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d7a11e-a9a1-4103-97d4-685fc52f8434-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.662239 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.662206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.662239 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.662236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.662384 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.662259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7v97\" (UniqueName: \"kubernetes.io/projected/f6d7a11e-a9a1-4103-97d4-685fc52f8434-kube-api-access-x7v97\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.662384 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.662291    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.753798 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.753760    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:07.762850 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.762825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.762958 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.762860    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d7a11e-a9a1-4103-97d4-685fc52f8434-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.762958 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.762882    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.762958 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.762918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.763139 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.763001    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7v97\" (UniqueName: \"kubernetes.io/projected/f6d7a11e-a9a1-4103-97d4-685fc52f8434-kube-api-access-x7v97\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.763139 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.763056    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.763139 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.763119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.765390 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.765371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6d7a11e-a9a1-4103-97d4-685fc52f8434-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.774858 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.774834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:07.774962 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:07.774899    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7v97\" (UniqueName: \"kubernetes.io/projected/f6d7a11e-a9a1-4103-97d4-685fc52f8434-kube-api-access-x7v97\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:08.422532 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:08.422497    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 22 21:06:08.425676 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:08.425636    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:08.549953 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:08.549923    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 22 21:06:08.555088 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:08.555065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:08.563453 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:08.563435    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 21:06:08.574074 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:08.574055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:08.763973 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:08.763901    2559 secret.go:189] Couldn't get secret openshift-ingress/istiod-tls: failed to sync secret cache: timed out waiting for the condition
Jul 22 21:06:08.764121 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:06:08.763994    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-dns-cert podName:f6d7a11e-a9a1-4103-97d4-685fc52f8434 nodeName:}" failed. No retries permitted until 2026-07-22 21:06:09.263972037 +0000 UTC m=+551.664774007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-csr-dns-cert" (UniqueName: "kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-dns-cert") pod "istiod-openshift-gateway-94bb8fbfd-9pxxm" (UID: "f6d7a11e-a9a1-4103-97d4-685fc52f8434") : failed to sync secret cache: timed out waiting for the condition
Jul 22 21:06:09.007032 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:09.007002    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-98c6m\""
Jul 22 21:06:09.107791 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:09.107719    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 22 21:06:09.275317 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:09.275275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:09.278077 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:09.278049    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6d7a11e-a9a1-4103-97d4-685fc52f8434-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9pxxm\" (UID: \"f6d7a11e-a9a1-4103-97d4-685fc52f8434\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:09.338523 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:09.338481    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:09.410520 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:09.410500    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:09.565583 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:09.565559    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"]
Jul 22 21:06:09.567835 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:06:09.567808    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d7a11e_a9a1_4103_97d4_685fc52f8434.slice/crio-8768449403e88856ccef7fee0f64f9d74f343efea7f445a6420cf7fecd9abd9d WatchSource:0}: Error finding container 8768449403e88856ccef7fee0f64f9d74f343efea7f445a6420cf7fecd9abd9d: Status 404 returned error can't find the container with id 8768449403e88856ccef7fee0f64f9d74f343efea7f445a6420cf7fecd9abd9d
Jul 22 21:06:10.014383 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:10.014353    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k" event={"ID":"b18e2476-a2d3-4c30-bf0b-464aafab41f6","Type":"ContainerStarted","Data":"856836e4155e9e17efcb2a4e9a9065225bf9b58efe72347af8d0959ddf00ac6a"}
Jul 22 21:06:10.015545 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:10.015520    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm" event={"ID":"f6d7a11e-a9a1-4103-97d4-685fc52f8434","Type":"ContainerStarted","Data":"8768449403e88856ccef7fee0f64f9d74f343efea7f445a6420cf7fecd9abd9d"}
Jul 22 21:06:10.040521 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:10.040477    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6766dd8-th24k" podStartSLOduration=2.086958602 podStartE2EDuration="5.040462738s" totalImagesPullingTime="2.953504136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:06:10.038813197 +0000 UTC m=+552.439615206" watchObservedRunningTime="2026-07-22 21:06:10.040462738 +0000 UTC m=+552.441264751"
Jul 22 21:06:11.327674 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.327637    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:11.486652 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.486621    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"]
Jul 22 21:06:11.491850 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.491829    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:11.501491 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.501461    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:11.502559 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.502534    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 22 21:06:11.502660 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.502566    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zcmng\""
Jul 22 21:06:11.509358 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.509338    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"]
Jul 22 21:06:11.593772 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.593701    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bf69893-ad00-45fa-9c76-a6b5dfffc588-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ss7nx\" (UID: \"6bf69893-ad00-45fa-9c76-a6b5dfffc588\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:11.593772 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.593746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79vqt\" (UniqueName: \"kubernetes.io/projected/6bf69893-ad00-45fa-9c76-a6b5dfffc588-kube-api-access-79vqt\") pod \"llmisvc-controller-manager-777c6bd7d5-ss7nx\" (UID: \"6bf69893-ad00-45fa-9c76-a6b5dfffc588\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:11.676092 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.676059    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:11.694509 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.694480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79vqt\" (UniqueName: \"kubernetes.io/projected/6bf69893-ad00-45fa-9c76-a6b5dfffc588-kube-api-access-79vqt\") pod \"llmisvc-controller-manager-777c6bd7d5-ss7nx\" (UID: \"6bf69893-ad00-45fa-9c76-a6b5dfffc588\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:11.694652 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.694582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bf69893-ad00-45fa-9c76-a6b5dfffc588-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ss7nx\" (UID: \"6bf69893-ad00-45fa-9c76-a6b5dfffc588\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:11.697252 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.697231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6bf69893-ad00-45fa-9c76-a6b5dfffc588-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ss7nx\" (UID: \"6bf69893-ad00-45fa-9c76-a6b5dfffc588\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:11.703415 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.703390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79vqt\" (UniqueName: \"kubernetes.io/projected/6bf69893-ad00-45fa-9c76-a6b5dfffc588-kube-api-access-79vqt\") pod \"llmisvc-controller-manager-777c6bd7d5-ss7nx\" (UID: \"6bf69893-ad00-45fa-9c76-a6b5dfffc588\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:11.805469 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:11.805431    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:12.080778 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:12.080743    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 21:06:12.080880 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:12.080808    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 22 21:06:12.174898 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:12.174868    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"]
Jul 22 21:06:12.177736 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:06:12.177712    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6bf69893_ad00_45fa_9c76_a6b5dfffc588.slice/crio-84c00c97edbf21de894cd0a94169fd31f19b5c082d2096a5225fa8dd68aad773 WatchSource:0}: Error finding container 84c00c97edbf21de894cd0a94169fd31f19b5c082d2096a5225fa8dd68aad773: Status 404 returned error can't find the container with id 84c00c97edbf21de894cd0a94169fd31f19b5c082d2096a5225fa8dd68aad773
Jul 22 21:06:12.849059 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:12.849023    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:13.034132 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.034093    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx" event={"ID":"6bf69893-ad00-45fa-9c76-a6b5dfffc588","Type":"ContainerStarted","Data":"84c00c97edbf21de894cd0a94169fd31f19b5c082d2096a5225fa8dd68aad773"}
Jul 22 21:06:13.036065 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.036027    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm" event={"ID":"f6d7a11e-a9a1-4103-97d4-685fc52f8434","Type":"ContainerStarted","Data":"e7b831fa93b3b25c85c979920396779c910f703966f58464802be659b84d74e1"}
Jul 22 21:06:13.036247 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.036223    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:13.038095 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.038063    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-9pxxm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 21:06:13.038217 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.038124    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm" podUID="f6d7a11e-a9a1-4103-97d4-685fc52f8434" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 21:06:13.065152 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.065127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:13.110563 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.110475    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm" podStartSLOduration=3.599935237 podStartE2EDuration="6.110459786s" totalImagesPullingTime="2.510524549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:06:13.109234987 +0000 UTC m=+555.510036991" watchObservedRunningTime="2026-07-22 21:06:13.110459786 +0000 UTC m=+555.511261768"
Jul 22 21:06:13.249910 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.249871    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:13.457208 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.457146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:13.787080 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:13.786984    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:14.023812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:14.023776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:14.041861 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:14.041794    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pxxm"
Jul 22 21:06:14.469935 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:14.469902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:14.599725 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:14.599689    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:16.051569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:16.051492    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx" event={"ID":"6bf69893-ad00-45fa-9c76-a6b5dfffc588","Type":"ContainerStarted","Data":"5575762138f8bc8dbe55d2b4d787e0898b9cc7eb3bbd4b9ce12002e58d90775b"}
Jul 22 21:06:16.051569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:16.051565    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:16.084590 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:16.084534    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx" podStartSLOduration=1.614610605 podStartE2EDuration="5.084512872s" totalImagesPullingTime="3.469902267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:06:16.081159609 +0000 UTC m=+558.481961611" watchObservedRunningTime="2026-07-22 21:06:16.084512872 +0000 UTC m=+558.485314856"
Jul 22 21:06:16.547973 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:16.547935    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:16.671956 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:16.671920    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:17.001896 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:17.001871    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fzq8x"
Jul 22 21:06:17.003753 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:17.003731    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lbs79"
Jul 22 21:06:46.223066 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:46.223027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:46.323593 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:46.323563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:06:47.059773 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:47.059748    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ss7nx"
Jul 22 21:06:57.966316 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:57.966288    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:06:57.967991 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:06:57.967970    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:07:01.359073 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:01.359037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:05.776226 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:05.776144    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:11.397129 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:11.397093    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:11.453414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:11.453383    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:22.637196 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:22.637105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:22.699969 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:22.699939    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:22.798563 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:22.798525    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:30.175036 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:30.174985    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:33.776596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:33.776560    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:36.806345 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:36.806314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:38.125902 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.125860    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"]
Jul 22 21:07:38.129489 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.129467    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.131836 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.131808    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 21:07:38.131836 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.131827    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 21:07:38.132015 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.131869    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tj6rc\""
Jul 22 21:07:38.137717 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.137668    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"]
Jul 22 21:07:38.183373 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.183330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/081d14f4-1898-4c70-95ef-977a984e7ae6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.183529 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.183404    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drxzb\" (UniqueName: \"kubernetes.io/projected/081d14f4-1898-4c70-95ef-977a984e7ae6-kube-api-access-drxzb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.238690 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.238656    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:38.284242 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.284215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drxzb\" (UniqueName: \"kubernetes.io/projected/081d14f4-1898-4c70-95ef-977a984e7ae6-kube-api-access-drxzb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.284411 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.284276    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/081d14f4-1898-4c70-95ef-977a984e7ae6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.284590 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.284571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/081d14f4-1898-4c70-95ef-977a984e7ae6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.313434 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.313416    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drxzb\" (UniqueName: \"kubernetes.io/projected/081d14f4-1898-4c70-95ef-977a984e7ae6-kube-api-access-drxzb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.441456 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.441433    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:38.581042 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.581014    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"]
Jul 22 21:07:38.584324 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:07:38.584297    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081d14f4_1898_4c70_95ef_977a984e7ae6.slice/crio-fa8985b5a60555787914d61aadbf193c5ee79cf7cdf3db4bb22fcab8b2366aaa WatchSource:0}: Error finding container fa8985b5a60555787914d61aadbf193c5ee79cf7cdf3db4bb22fcab8b2366aaa: Status 404 returned error can't find the container with id fa8985b5a60555787914d61aadbf193c5ee79cf7cdf3db4bb22fcab8b2366aaa
Jul 22 21:07:38.586759 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:38.586741    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 21:07:39.165488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:39.165455    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:39.371693 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:39.371659    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp" event={"ID":"081d14f4-1898-4c70-95ef-977a984e7ae6","Type":"ContainerStarted","Data":"fa8985b5a60555787914d61aadbf193c5ee79cf7cdf3db4bb22fcab8b2366aaa"}
Jul 22 21:07:41.361572 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:41.361535    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:41.765845 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:41.765817    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:42.120501 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:42.120419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:42.244452 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:42.244425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:44.390688 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.390656    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"]
Jul 22 21:07:44.393477 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.393456    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"
Jul 22 21:07:44.395880 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.395853    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hdd6w\""
Jul 22 21:07:44.404117 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.404092    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"]
Jul 22 21:07:44.431623 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.431597    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prcpn\" (UniqueName: \"kubernetes.io/projected/6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c-kube-api-access-prcpn\") pod \"authorino-operator-75cc8d5cb7-h57jl\" (UID: \"6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"
Jul 22 21:07:44.532871 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.532837    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prcpn\" (UniqueName: \"kubernetes.io/projected/6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c-kube-api-access-prcpn\") pod \"authorino-operator-75cc8d5cb7-h57jl\" (UID: \"6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"
Jul 22 21:07:44.541792 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.541764    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prcpn\" (UniqueName: \"kubernetes.io/projected/6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c-kube-api-access-prcpn\") pod \"authorino-operator-75cc8d5cb7-h57jl\" (UID: \"6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"
Jul 22 21:07:44.706695 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.706615    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"
Jul 22 21:07:44.811025 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:44.810988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:45.374725 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:45.374693    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"]
Jul 22 21:07:45.378145 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:07:45.378119    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5a3b6c_2dbe_40c4_8b85_f2ed16b10f1c.slice/crio-1fdfc7620f207d9458f3f23a516368c81a74ca4ef7bd76654cc60c1c497abfd6 WatchSource:0}: Error finding container 1fdfc7620f207d9458f3f23a516368c81a74ca4ef7bd76654cc60c1c497abfd6: Status 404 returned error can't find the container with id 1fdfc7620f207d9458f3f23a516368c81a74ca4ef7bd76654cc60c1c497abfd6
Jul 22 21:07:45.403046 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:45.403016    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp" event={"ID":"081d14f4-1898-4c70-95ef-977a984e7ae6","Type":"ContainerStarted","Data":"fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad"}
Jul 22 21:07:45.403654 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:45.403080    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:45.404284 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:45.404266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl" event={"ID":"6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c","Type":"ContainerStarted","Data":"1fdfc7620f207d9458f3f23a516368c81a74ca4ef7bd76654cc60c1c497abfd6"}
Jul 22 21:07:45.424107 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:45.424070    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp" podStartSLOduration=0.729882041 podStartE2EDuration="7.424057162s" totalImagesPullingTime="6.694175121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:07:45.422777541 +0000 UTC m=+647.823579554" watchObservedRunningTime="2026-07-22 21:07:45.424057162 +0000 UTC m=+647.824859144"
Jul 22 21:07:46.225986 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:46.225947    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:46.322777 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:46.322744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:47.615576 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:47.615538    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:48.347483 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:48.347445    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:48.417732 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:48.417702    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl" event={"ID":"6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c","Type":"ContainerStarted","Data":"81049c2c1b0f5eeac12ba6ad5ee8c43d65f6b868e92de2cc569f046d314d6923"}
Jul 22 21:07:48.417868 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:48.417840    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"
Jul 22 21:07:48.435826 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:48.435783    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl" podStartSLOduration=1.588811499 podStartE2EDuration="4.435765458s" totalImagesPullingTime="2.846953959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:07:48.434017722 +0000 UTC m=+650.834819718" watchObservedRunningTime="2026-07-22 21:07:48.435765458 +0000 UTC m=+650.836567440"
Jul 22 21:07:48.659741 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:48.659694    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:49.768824 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:49.768793    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:49.819014 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:49.818985    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:51.058533 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:51.058494    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:51.158750 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:51.158723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:56.409746 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:56.409713    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:57.252553 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:57.252525    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:57.365690 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:57.365649    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:07:58.081775 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.081746    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"]
Jul 22 21:07:58.082158 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.081906    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" containerName="manager" containerID="cri-o://fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad" gracePeriod=2
Jul 22 21:07:58.088266 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.088239    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"]
Jul 22 21:07:58.088902 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:58.088867    2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"081d14f4-1898-4c70-95ef-977a984e7ae6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-92blp\": pods \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:58.262153 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.262125    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"]
Jul 22 21:07:58.262500 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.262484    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" containerName="manager"
Jul 22 21:07:58.262581 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.262503    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" containerName="manager"
Jul 22 21:07:58.262634 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.262586    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" containerName="manager"
Jul 22 21:07:58.265251 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.265233    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.267340 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:58.267315    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:58.277633 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.277609    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"]
Jul 22 21:07:58.322696 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.322676    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:58.324749 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:58.324725    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:58.336083 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.336024    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prntj\" (UniqueName: \"kubernetes.io/projected/502c2e33-3356-4b74-bcac-a2cbcc262b9a-kube-api-access-prntj\") pod \"kuadrant-operator-controller-manager-86878c769f-cwwbl\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.336155 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.336101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/502c2e33-3356-4b74-bcac-a2cbcc262b9a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cwwbl\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.436893 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.436860    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drxzb\" (UniqueName: \"kubernetes.io/projected/081d14f4-1898-4c70-95ef-977a984e7ae6-kube-api-access-drxzb\") pod \"081d14f4-1898-4c70-95ef-977a984e7ae6\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") "
Jul 22 21:07:58.436989 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.436930    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/081d14f4-1898-4c70-95ef-977a984e7ae6-extensions-socket-volume\") pod \"081d14f4-1898-4c70-95ef-977a984e7ae6\" (UID: \"081d14f4-1898-4c70-95ef-977a984e7ae6\") "
Jul 22 21:07:58.437041 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.437026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/502c2e33-3356-4b74-bcac-a2cbcc262b9a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cwwbl\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.437078 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.437067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prntj\" (UniqueName: \"kubernetes.io/projected/502c2e33-3356-4b74-bcac-a2cbcc262b9a-kube-api-access-prntj\") pod \"kuadrant-operator-controller-manager-86878c769f-cwwbl\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.437365 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.437341    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/081d14f4-1898-4c70-95ef-977a984e7ae6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "081d14f4-1898-4c70-95ef-977a984e7ae6" (UID: "081d14f4-1898-4c70-95ef-977a984e7ae6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:07:58.437438 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.437422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/502c2e33-3356-4b74-bcac-a2cbcc262b9a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cwwbl\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.438921 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.438900    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/081d14f4-1898-4c70-95ef-977a984e7ae6-kube-api-access-drxzb" (OuterVolumeSpecName: "kube-api-access-drxzb") pod "081d14f4-1898-4c70-95ef-977a984e7ae6" (UID: "081d14f4-1898-4c70-95ef-977a984e7ae6"). InnerVolumeSpecName "kube-api-access-drxzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:07:58.446890 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.446868    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prntj\" (UniqueName: \"kubernetes.io/projected/502c2e33-3356-4b74-bcac-a2cbcc262b9a-kube-api-access-prntj\") pod \"kuadrant-operator-controller-manager-86878c769f-cwwbl\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.459148 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.459127    2559 generic.go:356] "Generic (PLEG): container finished" podID="081d14f4-1898-4c70-95ef-977a984e7ae6" containerID="fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad" exitCode=0
Jul 22 21:07:58.459243 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.459168    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:58.459243 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.459221    2559 scope.go:117] "RemoveContainer" containerID="fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad"
Jul 22 21:07:58.461402 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:58.461376    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:58.467414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.467390    2559 scope.go:117] "RemoveContainer" containerID="fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad"
Jul 22 21:07:58.467657 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:58.467637    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad\": container with ID starting with fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad not found: ID does not exist" containerID="fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad"
Jul 22 21:07:58.467707 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.467667    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad"} err="failed to get container status \"fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad\": rpc error: code = NotFound desc = could not find container \"fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad\": container with ID starting with fcb2c98724c1a74008fd7715d84abb725553428b87d0da62ad12d136be1991ad not found: ID does not exist"
Jul 22 21:07:58.469267 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:58.469249    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:58.537932 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.537909    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/081d14f4-1898-4c70-95ef-977a984e7ae6-extensions-socket-volume\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:07:58.537932 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.537930    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drxzb\" (UniqueName: \"kubernetes.io/projected/081d14f4-1898-4c70-95ef-977a984e7ae6-kube-api-access-drxzb\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:07:58.574897 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.574873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:58.703557 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:58.703500    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"]
Jul 22 21:07:58.705585 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:07:58.705554    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502c2e33_3356_4b74_bcac_a2cbcc262b9a.slice/crio-33b8418c72cbbbb689ec1de89af95892602aff9f328602c25052167925ee46eb WatchSource:0}: Error finding container 33b8418c72cbbbb689ec1de89af95892602aff9f328602c25052167925ee46eb: Status 404 returned error can't find the container with id 33b8418c72cbbbb689ec1de89af95892602aff9f328602c25052167925ee46eb
Jul 22 21:07:59.423488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:59.423456    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h57jl"
Jul 22 21:07:59.425348 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:59.425325    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:07:59.466418 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:59.466387    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl" event={"ID":"502c2e33-3356-4b74-bcac-a2cbcc262b9a","Type":"ContainerStarted","Data":"3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6"}
Jul 22 21:07:59.466596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:59.466441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl" event={"ID":"502c2e33-3356-4b74-bcac-a2cbcc262b9a","Type":"ContainerStarted","Data":"33b8418c72cbbbb689ec1de89af95892602aff9f328602c25052167925ee46eb"}
Jul 22 21:07:59.466596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:59.466469    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:07:59.486442 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:07:59.486395    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl" podStartSLOduration=1.486376052 podStartE2EDuration="1.486376052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:07:59.484694611 +0000 UTC m=+661.885496607" watchObservedRunningTime="2026-07-22 21:07:59.486376052 +0000 UTC m=+661.887178035"
Jul 22 21:07:59.486589 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:07:59.486438    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-92blp\" is forbidden: User \"system:node:ip-10-0-131-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-38.ec2.internal' and this object" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-92blp"
Jul 22 21:08:00.076552 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:00.076517    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="081d14f4-1898-4c70-95ef-977a984e7ae6" path="/var/lib/kubelet/pods/081d14f4-1898-4c70-95ef-977a984e7ae6/volumes"
Jul 22 21:08:01.999894 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:01.999861    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:02.099612 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:02.099577    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:04.506371 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:04.506333    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:04.604048 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:04.604019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:05.271595 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:05.271567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:07.875447 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:07.875407    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:07.975787 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:07.975753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:10.472819 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:10.472787    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:08:14.921348 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:14.921317    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:15.026285 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:15.026254    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:19.750820 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:19.750777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:19.807784 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:19.807741    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:22.322596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:22.322563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:22.423595 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:22.423568    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:25.920020 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:25.919967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:26.020697 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:26.020657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:26.846619 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:26.846563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:26.989036 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:26.989002    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:28.874198 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:28.874142    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:28.996414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:28.996381    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:29.097554 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:29.097516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:29.744108 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:29.744073    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:30.333542 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:30.333503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:34.325559 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:34.325527    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:34.426822 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:34.426793    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:39.120095 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:39.120054    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:39.221390 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:39.221364    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:50.158048 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:50.157954    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:50.218361 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:50.218326    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:08:50.658876 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:08:50.658837    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:03.385485 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:03.385452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:03.556161 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:03.556128    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:10.775016 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.774978    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:10.793031 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.793001    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c4f7fb667-h6tcw"]
Jul 22 21:09:10.796287 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.796267    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:10.798154 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.798126    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9ggqk\""
Jul 22 21:09:10.798346 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.798324    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 21:09:10.798783 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.798766    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 21:09:10.805023 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.805000    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4f7fb667-h6tcw"]
Jul 22 21:09:10.936340 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.936314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert\") pod \"maas-controller-c4f7fb667-h6tcw\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") " pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:10.936462 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.936350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67m6v\" (UniqueName: \"kubernetes.io/projected/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-kube-api-access-67m6v\") pod \"maas-controller-c4f7fb667-h6tcw\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") " pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:10.954055 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.954033    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d5898b76-lrgvq"]
Jul 22 21:09:10.956186 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.956161    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:10.967520 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.967496    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:10.967960 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:10.967943    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d5898b76-lrgvq"]
Jul 22 21:09:11.024521 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.024498    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:11.037619 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.037570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67m6v\" (UniqueName: \"kubernetes.io/projected/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-kube-api-access-67m6v\") pod \"maas-controller-c4f7fb667-h6tcw\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") " pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:11.037726 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.037632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert\") pod \"maas-controller-c4f7fb667-h6tcw\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") " pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:11.037726 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:09:11.037714    2559 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 22 21:09:11.037833 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:09:11.037768    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert podName:7df06001-64d5-40a1-a8ad-ea7ba51c50e2 nodeName:}" failed. No retries permitted until 2026-07-22 21:09:11.537752988 +0000 UTC m=+733.938554949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert") pod "maas-controller-c4f7fb667-h6tcw" (UID: "7df06001-64d5-40a1-a8ad-ea7ba51c50e2") : secret "maas-controller-webhook-cert" not found
Jul 22 21:09:11.045525 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.045505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67m6v\" (UniqueName: \"kubernetes.io/projected/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-kube-api-access-67m6v\") pod \"maas-controller-c4f7fb667-h6tcw\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") " pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:11.126642 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.126612    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:11.138694 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.138668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-webhook-cert\") pod \"maas-controller-6d5898b76-lrgvq\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") " pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:11.138770 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.138707    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xtl4\" (UniqueName: \"kubernetes.io/projected/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-kube-api-access-6xtl4\") pod \"maas-controller-6d5898b76-lrgvq\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") " pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:11.239975 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.239957    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-webhook-cert\") pod \"maas-controller-6d5898b76-lrgvq\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") " pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:11.240059 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.239988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xtl4\" (UniqueName: \"kubernetes.io/projected/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-kube-api-access-6xtl4\") pod \"maas-controller-6d5898b76-lrgvq\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") " pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:11.242213 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.242170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-webhook-cert\") pod \"maas-controller-6d5898b76-lrgvq\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") " pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:11.247485 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.247458    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xtl4\" (UniqueName: \"kubernetes.io/projected/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-kube-api-access-6xtl4\") pod \"maas-controller-6d5898b76-lrgvq\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") " pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:11.265230 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.265213    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:11.386541 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.386518    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d5898b76-lrgvq"]
Jul 22 21:09:11.389230 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:09:11.389203    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5018584_fed4_4a30_9dfb_bbd36b80a9b8.slice/crio-0474a42966c237caf885daec499f277635ddd1864757b443775a8eb9c0cf8411 WatchSource:0}: Error finding container 0474a42966c237caf885daec499f277635ddd1864757b443775a8eb9c0cf8411: Status 404 returned error can't find the container with id 0474a42966c237caf885daec499f277635ddd1864757b443775a8eb9c0cf8411
Jul 22 21:09:11.543679 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.543653    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert\") pod \"maas-controller-c4f7fb667-h6tcw\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") " pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:11.545805 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.545779    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert\") pod \"maas-controller-c4f7fb667-h6tcw\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") " pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:11.706673 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.706653    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:11.781263 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.781229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-lrgvq" event={"ID":"e5018584-fed4-4a30-9dfb-bbd36b80a9b8","Type":"ContainerStarted","Data":"0474a42966c237caf885daec499f277635ddd1864757b443775a8eb9c0cf8411"}
Jul 22 21:09:11.831152 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:11.831128    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4f7fb667-h6tcw"]
Jul 22 21:09:11.833501 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:09:11.833475    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df06001_64d5_40a1_a8ad_ea7ba51c50e2.slice/crio-05d999cb0c3aeabe64a77bda24ea6a22d9c03e875116959d566c9935abb458be WatchSource:0}: Error finding container 05d999cb0c3aeabe64a77bda24ea6a22d9c03e875116959d566c9935abb458be: Status 404 returned error can't find the container with id 05d999cb0c3aeabe64a77bda24ea6a22d9c03e875116959d566c9935abb458be
Jul 22 21:09:12.792506 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:12.792469    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4f7fb667-h6tcw" event={"ID":"7df06001-64d5-40a1-a8ad-ea7ba51c50e2","Type":"ContainerStarted","Data":"05d999cb0c3aeabe64a77bda24ea6a22d9c03e875116959d566c9935abb458be"}
Jul 22 21:09:14.122603 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.122567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:14.236619 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.236530    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:14.805424 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.805383    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4f7fb667-h6tcw" event={"ID":"7df06001-64d5-40a1-a8ad-ea7ba51c50e2","Type":"ContainerStarted","Data":"0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79"}
Jul 22 21:09:14.805617 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.805518    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:14.806942 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.806910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-lrgvq" event={"ID":"e5018584-fed4-4a30-9dfb-bbd36b80a9b8","Type":"ContainerStarted","Data":"e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533"}
Jul 22 21:09:14.807050 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.806957    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:14.822062 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.822018    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c4f7fb667-h6tcw" podStartSLOduration=2.689814292 podStartE2EDuration="4.822006149s" totalImagesPullingTime="2.132191857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:09:14.821025304 +0000 UTC m=+737.221827302" watchObservedRunningTime="2026-07-22 21:09:14.822006149 +0000 UTC m=+737.222808184"
Jul 22 21:09:14.836416 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:14.836382    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d5898b76-lrgvq" podStartSLOduration=2.24979351 podStartE2EDuration="4.836370127s" totalImagesPullingTime="2.586576617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:09:14.8355377 +0000 UTC m=+737.236339714" watchObservedRunningTime="2026-07-22 21:09:14.836370127 +0000 UTC m=+737.237172111"
Jul 22 21:09:25.813451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:25.813423    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:25.813782 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:25.813488    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:09:25.879276 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:25.879246    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c4f7fb667-h6tcw"]
Jul 22 21:09:25.879495 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:25.879471    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-c4f7fb667-h6tcw" podUID="7df06001-64d5-40a1-a8ad-ea7ba51c50e2" containerName="manager" containerID="cri-o://0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79" gracePeriod=30
Jul 22 21:09:26.122207 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.122166    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:26.147424 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.147397    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67m6v\" (UniqueName: \"kubernetes.io/projected/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-kube-api-access-67m6v\") pod \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") "
Jul 22 21:09:26.147563 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.147464    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert\") pod \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\" (UID: \"7df06001-64d5-40a1-a8ad-ea7ba51c50e2\") "
Jul 22 21:09:26.149742 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.149700    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7df06001-64d5-40a1-a8ad-ea7ba51c50e2" (UID: "7df06001-64d5-40a1-a8ad-ea7ba51c50e2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:09:26.149887 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.149747    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-kube-api-access-67m6v" (OuterVolumeSpecName: "kube-api-access-67m6v") pod "7df06001-64d5-40a1-a8ad-ea7ba51c50e2" (UID: "7df06001-64d5-40a1-a8ad-ea7ba51c50e2"). InnerVolumeSpecName "kube-api-access-67m6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:09:26.248924 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.248902    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67m6v\" (UniqueName: \"kubernetes.io/projected/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-kube-api-access-67m6v\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:09:26.248924 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.248925    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7df06001-64d5-40a1-a8ad-ea7ba51c50e2-webhook-cert\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:09:26.859522 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.859487    2559 generic.go:356] "Generic (PLEG): container finished" podID="7df06001-64d5-40a1-a8ad-ea7ba51c50e2" containerID="0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79" exitCode=0
Jul 22 21:09:26.859948 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.859538    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4f7fb667-h6tcw"
Jul 22 21:09:26.859948 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.859559    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4f7fb667-h6tcw" event={"ID":"7df06001-64d5-40a1-a8ad-ea7ba51c50e2","Type":"ContainerDied","Data":"0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79"}
Jul 22 21:09:26.859948 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.859606    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4f7fb667-h6tcw" event={"ID":"7df06001-64d5-40a1-a8ad-ea7ba51c50e2","Type":"ContainerDied","Data":"05d999cb0c3aeabe64a77bda24ea6a22d9c03e875116959d566c9935abb458be"}
Jul 22 21:09:26.859948 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.859628    2559 scope.go:117] "RemoveContainer" containerID="0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79"
Jul 22 21:09:26.873443 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.873423    2559 scope.go:117] "RemoveContainer" containerID="0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79"
Jul 22 21:09:26.873702 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:09:26.873686    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79\": container with ID starting with 0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79 not found: ID does not exist" containerID="0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79"
Jul 22 21:09:26.873757 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.873708    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79"} err="failed to get container status \"0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79\": rpc error: code = NotFound desc = could not find container \"0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79\": container with ID starting with 0a9c3ff834c604626922d4e26702839c9a56f89f64af2a0dd216f570ce5edb79 not found: ID does not exist"
Jul 22 21:09:26.882516 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.882495    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c4f7fb667-h6tcw"]
Jul 22 21:09:26.886245 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:26.886224    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c4f7fb667-h6tcw"]
Jul 22 21:09:28.076811 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:28.076777    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7df06001-64d5-40a1-a8ad-ea7ba51c50e2" path="/var/lib/kubelet/pods/7df06001-64d5-40a1-a8ad-ea7ba51c50e2/volumes"
Jul 22 21:09:39.119542 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:39.119509    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:46.877294 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:46.877247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:46.967081 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:46.967046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:47.596316 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.596277    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:47.698608 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.698572    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"]
Jul 22 21:09:47.698972 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.698953    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df06001-64d5-40a1-a8ad-ea7ba51c50e2" containerName="manager"
Jul 22 21:09:47.699058 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.698973    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df06001-64d5-40a1-a8ad-ea7ba51c50e2" containerName="manager"
Jul 22 21:09:47.699111 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.699065    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df06001-64d5-40a1-a8ad-ea7ba51c50e2" containerName="manager"
Jul 22 21:09:47.703761 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.703742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.705868 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.705850    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 22 21:09:47.706101 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.706088    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lbfbx\""
Jul 22 21:09:47.706563 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.706547    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 21:09:47.706654 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.706571    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 21:09:47.712997 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.712975    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"]
Jul 22 21:09:47.802800 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.802770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f384b6b-3afb-40a8-b86b-47bfd1578dad-openshift-service-ca\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.802952 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.802840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gv6f\" (UniqueName: \"kubernetes.io/projected/0f384b6b-3afb-40a8-b86b-47bfd1578dad-kube-api-access-9gv6f\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.802952 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.802867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f384b6b-3afb-40a8-b86b-47bfd1578dad-maas-api-tls\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.903929 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.903903    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f384b6b-3afb-40a8-b86b-47bfd1578dad-openshift-service-ca\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.904256 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.903963    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv6f\" (UniqueName: \"kubernetes.io/projected/0f384b6b-3afb-40a8-b86b-47bfd1578dad-kube-api-access-9gv6f\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.904256 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.903988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f384b6b-3afb-40a8-b86b-47bfd1578dad-maas-api-tls\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.904652 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.904628    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f384b6b-3afb-40a8-b86b-47bfd1578dad-openshift-service-ca\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.906289 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.906271    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f384b6b-3afb-40a8-b86b-47bfd1578dad-maas-api-tls\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.912329 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.912309    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gv6f\" (UniqueName: \"kubernetes.io/projected/0f384b6b-3afb-40a8-b86b-47bfd1578dad-kube-api-access-9gv6f\") pod \"maas-api-859cb6f49-tzkpt\" (UID: \"0f384b6b-3afb-40a8-b86b-47bfd1578dad\") " pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:47.922349 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:47.922319    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:48.015009 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.014981    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:48.151873 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.151791    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"]
Jul 22 21:09:48.154816 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:09:48.154756    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f384b6b_3afb_40a8_b86b_47bfd1578dad.slice/crio-ed2f8be73069fe57fda03688f93102f1a15d5c3968a5a475ad76db11e31e8113 WatchSource:0}: Error finding container ed2f8be73069fe57fda03688f93102f1a15d5c3968a5a475ad76db11e31e8113: Status 404 returned error can't find the container with id ed2f8be73069fe57fda03688f93102f1a15d5c3968a5a475ad76db11e31e8113
Jul 22 21:09:48.260397 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.260365    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:48.362025 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.361993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:48.362520 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.362503    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-r4486"]
Jul 22 21:09:48.365431 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.365414    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.368188 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.368156    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 22 21:09:48.368280 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.368199    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-c8w9w\""
Jul 22 21:09:48.376600 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.376579    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-r4486"]
Jul 22 21:09:48.400248 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.400218    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hbp5n"]
Jul 22 21:09:48.402590 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.402574    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.407413 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.407354    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8h5d\" (UniqueName: \"kubernetes.io/projected/401d4b9c-7c49-4975-a310-aa2c958a7a5f-kube-api-access-r8h5d\") pod \"payload-pre-processing-7677b774bb-r4486\" (UID: \"401d4b9c-7c49-4975-a310-aa2c958a7a5f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.407492 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.407460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/401d4b9c-7c49-4975-a310-aa2c958a7a5f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-r4486\" (UID: \"401d4b9c-7c49-4975-a310-aa2c958a7a5f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.416431 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.416412    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hbp5n"]
Jul 22 21:09:48.508841 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.508805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/401d4b9c-7c49-4975-a310-aa2c958a7a5f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-r4486\" (UID: \"401d4b9c-7c49-4975-a310-aa2c958a7a5f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.509008 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.508858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cbce6c2-b781-4112-b55e-ca19806a301f-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hbp5n\" (UID: \"0cbce6c2-b781-4112-b55e-ca19806a301f\") " pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.509008 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.508890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9c58\" (UniqueName: \"kubernetes.io/projected/0cbce6c2-b781-4112-b55e-ca19806a301f-kube-api-access-c9c58\") pod \"payload-processing-5bf47cd988-hbp5n\" (UID: \"0cbce6c2-b781-4112-b55e-ca19806a301f\") " pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.509008 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.508973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8h5d\" (UniqueName: \"kubernetes.io/projected/401d4b9c-7c49-4975-a310-aa2c958a7a5f-kube-api-access-r8h5d\") pod \"payload-pre-processing-7677b774bb-r4486\" (UID: \"401d4b9c-7c49-4975-a310-aa2c958a7a5f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.509569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.509553    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/401d4b9c-7c49-4975-a310-aa2c958a7a5f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-r4486\" (UID: \"401d4b9c-7c49-4975-a310-aa2c958a7a5f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.521820 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.521795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8h5d\" (UniqueName: \"kubernetes.io/projected/401d4b9c-7c49-4975-a310-aa2c958a7a5f-kube-api-access-r8h5d\") pod \"payload-pre-processing-7677b774bb-r4486\" (UID: \"401d4b9c-7c49-4975-a310-aa2c958a7a5f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.609587 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.609555    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cbce6c2-b781-4112-b55e-ca19806a301f-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hbp5n\" (UID: \"0cbce6c2-b781-4112-b55e-ca19806a301f\") " pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.609587 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.609589    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9c58\" (UniqueName: \"kubernetes.io/projected/0cbce6c2-b781-4112-b55e-ca19806a301f-kube-api-access-c9c58\") pod \"payload-processing-5bf47cd988-hbp5n\" (UID: \"0cbce6c2-b781-4112-b55e-ca19806a301f\") " pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.610281 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.610263    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cbce6c2-b781-4112-b55e-ca19806a301f-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hbp5n\" (UID: \"0cbce6c2-b781-4112-b55e-ca19806a301f\") " pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.618172 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.618148    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9c58\" (UniqueName: \"kubernetes.io/projected/0cbce6c2-b781-4112-b55e-ca19806a301f-kube-api-access-c9c58\") pod \"payload-processing-5bf47cd988-hbp5n\" (UID: \"0cbce6c2-b781-4112-b55e-ca19806a301f\") " pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.676319 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.676252    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:48.712571 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.712517    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:48.825911 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.825883    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-r4486"]
Jul 22 21:09:48.828422 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:09:48.828389    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401d4b9c_7c49_4975_a310_aa2c958a7a5f.slice/crio-9bdc6ff35a77b4013c0b24886b0edca6216fadf5b7f862551ffef1787f43c42f WatchSource:0}: Error finding container 9bdc6ff35a77b4013c0b24886b0edca6216fadf5b7f862551ffef1787f43c42f: Status 404 returned error can't find the container with id 9bdc6ff35a77b4013c0b24886b0edca6216fadf5b7f862551ffef1787f43c42f
Jul 22 21:09:48.858590 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.858569    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hbp5n"]
Jul 22 21:09:48.860431 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:09:48.860403    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbce6c2_b781_4112_b55e_ca19806a301f.slice/crio-06e350383c103a4653465a0017bf717f57c42859e9ac6e32403c5229b172f506 WatchSource:0}: Error finding container 06e350383c103a4653465a0017bf717f57c42859e9ac6e32403c5229b172f506: Status 404 returned error can't find the container with id 06e350383c103a4653465a0017bf717f57c42859e9ac6e32403c5229b172f506
Jul 22 21:09:48.954561 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.954484    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt" event={"ID":"0f384b6b-3afb-40a8-b86b-47bfd1578dad","Type":"ContainerStarted","Data":"ed2f8be73069fe57fda03688f93102f1a15d5c3968a5a475ad76db11e31e8113"}
Jul 22 21:09:48.955890 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.955863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486" event={"ID":"401d4b9c-7c49-4975-a310-aa2c958a7a5f","Type":"ContainerStarted","Data":"9bdc6ff35a77b4013c0b24886b0edca6216fadf5b7f862551ffef1787f43c42f"}
Jul 22 21:09:48.957200 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:48.957127    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n" event={"ID":"0cbce6c2-b781-4112-b55e-ca19806a301f","Type":"ContainerStarted","Data":"06e350383c103a4653465a0017bf717f57c42859e9ac6e32403c5229b172f506"}
Jul 22 21:09:51.983385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:51.983354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486" event={"ID":"401d4b9c-7c49-4975-a310-aa2c958a7a5f","Type":"ContainerStarted","Data":"a6ce7ec9fe22de9a06d49171314dfae6b58e7f93c25bf0ecafea01fcbb5a37fc"}
Jul 22 21:09:51.983846 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:51.983443    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:09:51.984923 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:51.984897    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n" event={"ID":"0cbce6c2-b781-4112-b55e-ca19806a301f","Type":"ContainerStarted","Data":"e6762f46b4358bce1ad53a990783f6431c883f325ad520e65a3cdaa84e36d61b"}
Jul 22 21:09:51.985045 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:51.984947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:09:51.986329 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:51.986309    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt" event={"ID":"0f384b6b-3afb-40a8-b86b-47bfd1578dad","Type":"ContainerStarted","Data":"0ed7a4cc8ae63049a406121825d16415d75e4472f3e380c1aeb81ecb49f961be"}
Jul 22 21:09:51.986425 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:51.986360    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:52.001116 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:52.001077    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486" podStartSLOduration=1.887859078 podStartE2EDuration="4.001063686s" totalImagesPullingTime="2.113204608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:09:51.999629904 +0000 UTC m=+774.400431887" watchObservedRunningTime="2026-07-22 21:09:52.001063686 +0000 UTC m=+774.401865668"
Jul 22 21:09:52.016302 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:52.016262    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n" podStartSLOduration=1.943363583 podStartE2EDuration="4.016250039s" totalImagesPullingTime="2.072886456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:09:52.014973321 +0000 UTC m=+774.415775305" watchObservedRunningTime="2026-07-22 21:09:52.016250039 +0000 UTC m=+774.417052021"
Jul 22 21:09:52.032573 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:52.032530    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt" podStartSLOduration=2.287163654 podStartE2EDuration="5.032519409s" totalImagesPullingTime="2.745355755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:09:52.032021842 +0000 UTC m=+774.432823825" watchObservedRunningTime="2026-07-22 21:09:52.032519409 +0000 UTC m=+774.433321395"
Jul 22 21:09:57.995713 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:57.995689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-859cb6f49-tzkpt"
Jul 22 21:09:58.961960 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:58.961916    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:09:59.059897 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:09:59.059864    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:10:02.992863 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:02.992832    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-hbp5n"
Jul 22 21:10:02.993308 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:02.992910    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-r4486"
Jul 22 21:10:33.924747 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:33.924708    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:10:45.210321 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.210288    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"]
Jul 22 21:10:45.214121 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.214098    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.217106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.217084    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 21:10:45.217106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.217100    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 21:10:45.217303 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.217083    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 22 21:10:45.217303 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.217083    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s7f6b\""
Jul 22 21:10:45.225100 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.225078    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"]
Jul 22 21:10:45.330271 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.330241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjpx8\" (UniqueName: \"kubernetes.io/projected/deab4530-6f24-409c-9bf0-804f2e0d9591-kube-api-access-pjpx8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.330406 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.330329    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.330406 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.330374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.330502 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.330410    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.330502 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.330442    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.330571 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.330496    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deab4530-6f24-409c-9bf0-804f2e0d9591-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431549 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431407    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431549 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431549 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431476    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deab4530-6f24-409c-9bf0-804f2e0d9591-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431723 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjpx8\" (UniqueName: \"kubernetes.io/projected/deab4530-6f24-409c-9bf0-804f2e0d9591-kube-api-access-pjpx8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431723 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431833 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431779    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.431833 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.431796    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.432042 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.432023    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.433666 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.433642    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/deab4530-6f24-409c-9bf0-804f2e0d9591-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.433893 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.433874    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/deab4530-6f24-409c-9bf0-804f2e0d9591-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.439108 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.439090    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjpx8\" (UniqueName: \"kubernetes.io/projected/deab4530-6f24-409c-9bf0-804f2e0d9591-kube-api-access-pjpx8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-57spt\" (UID: \"deab4530-6f24-409c-9bf0-804f2e0d9591\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.528264 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.528203    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:45.662716 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.662693    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"]
Jul 22 21:10:45.665057 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:10:45.665029    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeab4530_6f24_409c_9bf0_804f2e0d9591.slice/crio-46e6052b0266f4c2020dde79f5860d5903ff40b4e7404ef8d75f323704a9d84e WatchSource:0}: Error finding container 46e6052b0266f4c2020dde79f5860d5903ff40b4e7404ef8d75f323704a9d84e: Status 404 returned error can't find the container with id 46e6052b0266f4c2020dde79f5860d5903ff40b4e7404ef8d75f323704a9d84e
Jul 22 21:10:45.691519 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:45.691477    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:10:46.232867 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:46.232828    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt" event={"ID":"deab4530-6f24-409c-9bf0-804f2e0d9591","Type":"ContainerStarted","Data":"46e6052b0266f4c2020dde79f5860d5903ff40b4e7404ef8d75f323704a9d84e"}
Jul 22 21:10:47.245280 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:47.245242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt" event={"ID":"deab4530-6f24-409c-9bf0-804f2e0d9591","Type":"ContainerStarted","Data":"277abf016ecdd97b9959365c6cab5a23590d1daefaa7cab3e5c2a410f43c883d"}
Jul 22 21:10:47.264931 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:47.264880    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt" podStartSLOduration=1.261611688 podStartE2EDuration="2.264866498s" totalImagesPullingTime="1.00325481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:10:47.263543332 +0000 UTC m=+829.664345359" watchObservedRunningTime="2026-07-22 21:10:47.264866498 +0000 UTC m=+829.665668495"
Jul 22 21:10:51.598808 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:51.598776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:10:55.529362 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:55.529331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:55.529828 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:55.529681    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:55.541758 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:55.541730    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:10:55.545882 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:55.545853    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:10:56.295154 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:10:56.295123    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-57spt"
Jul 22 21:11:08.499143 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:08.499106    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:11:21.292154 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.292114    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"]
Jul 22 21:11:21.296828 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.296803    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.299296 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.299278    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 22 21:11:21.305858 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.305836    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"]
Jul 22 21:11:21.329458 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.329416    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:11:21.391800 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.391780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.391911 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.391812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629f7b9a-cb35-444d-94cd-33de243a0dcc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.391911 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.391866    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.391911 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.391895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.392052 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.391967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nvz2\" (UniqueName: \"kubernetes.io/projected/629f7b9a-cb35-444d-94cd-33de243a0dcc-kube-api-access-7nvz2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.392052 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.392010    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.492899 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.492873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493018 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.492916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvz2\" (UniqueName: \"kubernetes.io/projected/629f7b9a-cb35-444d-94cd-33de243a0dcc-kube-api-access-7nvz2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493018 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.492937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493018 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.492958    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493018 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.492977    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629f7b9a-cb35-444d-94cd-33de243a0dcc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493167 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.493031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493358 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.493333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493404 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.493350    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.493440 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.493394    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.495301 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.495285    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629f7b9a-cb35-444d-94cd-33de243a0dcc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.495481 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.495464    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629f7b9a-cb35-444d-94cd-33de243a0dcc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.509940 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.509917    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nvz2\" (UniqueName: \"kubernetes.io/projected/629f7b9a-cb35-444d-94cd-33de243a0dcc-kube-api-access-7nvz2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qcbhh\" (UID: \"629f7b9a-cb35-444d-94cd-33de243a0dcc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.608274 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.608211    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:21.736665 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:21.736642    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"]
Jul 22 21:11:21.738740 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:11:21.738714    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629f7b9a_cb35_444d_94cd_33de243a0dcc.slice/crio-2ca1d22ebb20f5a971b87e207c3dc95380c82ad50ab807f24c78f50abf1d63de WatchSource:0}: Error finding container 2ca1d22ebb20f5a971b87e207c3dc95380c82ad50ab807f24c78f50abf1d63de: Status 404 returned error can't find the container with id 2ca1d22ebb20f5a971b87e207c3dc95380c82ad50ab807f24c78f50abf1d63de
Jul 22 21:11:22.401254 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:22.401225    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh" event={"ID":"629f7b9a-cb35-444d-94cd-33de243a0dcc","Type":"ContainerStarted","Data":"c5c730783dc047c270c11f1e49aac07677830b466090cd966f0fe43ced87ebed"}
Jul 22 21:11:22.401604 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:22.401285    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh" event={"ID":"629f7b9a-cb35-444d-94cd-33de243a0dcc","Type":"ContainerStarted","Data":"2ca1d22ebb20f5a971b87e207c3dc95380c82ad50ab807f24c78f50abf1d63de"}
Jul 22 21:11:22.417453 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:22.417403    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh" podStartSLOduration=1.200578069 podStartE2EDuration="1.417386272s" totalImagesPullingTime="216.808203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:11:22.416155642 +0000 UTC m=+864.816957644" watchObservedRunningTime="2026-07-22 21:11:22.417386272 +0000 UTC m=+864.818188275"
Jul 22 21:11:22.752081 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:22.751979    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:11:22.806131 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:22.806095    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:11:31.608383 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:31.608349    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:31.608739 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:31.608435    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:31.620454 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:31.620433    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:32.460605 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:32.460579    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qcbhh"
Jul 22 21:11:39.327751 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:39.327714    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:11:45.508481 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:45.508449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:11:45.559478 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:45.559445    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:11:57.996677 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:57.996646    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:11:57.997753 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:11:57.997734    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:12:43.261520 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:12:43.261476    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:12:44.469016 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:12:44.468982    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:12:47.593917 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:12:47.593883    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:12:47.693523 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:12:47.693497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:13:30.895021 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:30.894977    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d5898b76-lrgvq"]
Jul 22 21:13:30.895592 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:30.895209    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d5898b76-lrgvq" podUID="e5018584-fed4-4a30-9dfb-bbd36b80a9b8" containerName="manager" containerID="cri-o://e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533" gracePeriod=30
Jul 22 21:13:31.147311 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:31.147256    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:13:31.232851 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:31.232819    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xtl4\" (UniqueName: \"kubernetes.io/projected/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-kube-api-access-6xtl4\") pod \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") "
Jul 22 21:13:31.233006 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:31.232859    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-webhook-cert\") pod \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\" (UID: \"e5018584-fed4-4a30-9dfb-bbd36b80a9b8\") "
Jul 22 21:13:31.234771 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:31.234746    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e5018584-fed4-4a30-9dfb-bbd36b80a9b8" (UID: "e5018584-fed4-4a30-9dfb-bbd36b80a9b8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:13:31.234771 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:31.234745    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-kube-api-access-6xtl4" (OuterVolumeSpecName: "kube-api-access-6xtl4") pod "e5018584-fed4-4a30-9dfb-bbd36b80a9b8" (UID: "e5018584-fed4-4a30-9dfb-bbd36b80a9b8"). InnerVolumeSpecName "kube-api-access-6xtl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:13:31.334065 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:31.334035    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xtl4\" (UniqueName: \"kubernetes.io/projected/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-kube-api-access-6xtl4\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:13:31.334065 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:31.334061    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5018584-fed4-4a30-9dfb-bbd36b80a9b8-webhook-cert\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:13:32.000210 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.000159    2559 generic.go:356] "Generic (PLEG): container finished" podID="e5018584-fed4-4a30-9dfb-bbd36b80a9b8" containerID="e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533" exitCode=0
Jul 22 21:13:32.000653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.000213    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-lrgvq" event={"ID":"e5018584-fed4-4a30-9dfb-bbd36b80a9b8","Type":"ContainerDied","Data":"e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533"}
Jul 22 21:13:32.000653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.000264    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-lrgvq" event={"ID":"e5018584-fed4-4a30-9dfb-bbd36b80a9b8","Type":"ContainerDied","Data":"0474a42966c237caf885daec499f277635ddd1864757b443775a8eb9c0cf8411"}
Jul 22 21:13:32.000653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.000219    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-lrgvq"
Jul 22 21:13:32.000653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.000296    2559 scope.go:117] "RemoveContainer" containerID="e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533"
Jul 22 21:13:32.010201 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.010163    2559 scope.go:117] "RemoveContainer" containerID="e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533"
Jul 22 21:13:32.010433 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:13:32.010415    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533\": container with ID starting with e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533 not found: ID does not exist" containerID="e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533"
Jul 22 21:13:32.010490 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.010439    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533"} err="failed to get container status \"e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533\": rpc error: code = NotFound desc = could not find container \"e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533\": container with ID starting with e3e75af3548d509b685c965bf80c365c2fe4764a2ad6de3b357c7bfd7b64d533 not found: ID does not exist"
Jul 22 21:13:32.022161 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.022142    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d5898b76-lrgvq"]
Jul 22 21:13:32.026544 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.026523    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d5898b76-lrgvq"]
Jul 22 21:13:32.076868 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:13:32.076841    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5018584-fed4-4a30-9dfb-bbd36b80a9b8" path="/var/lib/kubelet/pods/e5018584-fed4-4a30-9dfb-bbd36b80a9b8/volumes"
Jul 22 21:14:32.859757 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.859721    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d5898b76-x5w8m"]
Jul 22 21:14:32.860228 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.860115    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5018584-fed4-4a30-9dfb-bbd36b80a9b8" containerName="manager"
Jul 22 21:14:32.860228 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.860128    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5018584-fed4-4a30-9dfb-bbd36b80a9b8" containerName="manager"
Jul 22 21:14:32.860228 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.860205    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5018584-fed4-4a30-9dfb-bbd36b80a9b8" containerName="manager"
Jul 22 21:14:32.863128 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.863113    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:32.866059 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.866040    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 21:14:32.866164 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.866090    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 21:14:32.866164 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.866040    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9ggqk\""
Jul 22 21:14:32.873439 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.873417    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d5898b76-x5w8m"]
Jul 22 21:14:32.981935 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.981905    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvw56\" (UniqueName: \"kubernetes.io/projected/2efeab65-f83c-4ea6-ad99-03abb31788c9-kube-api-access-nvw56\") pod \"maas-controller-6d5898b76-x5w8m\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") " pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:32.982045 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:32.981941    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2efeab65-f83c-4ea6-ad99-03abb31788c9-webhook-cert\") pod \"maas-controller-6d5898b76-x5w8m\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") " pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:33.083284 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:33.083254    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2efeab65-f83c-4ea6-ad99-03abb31788c9-webhook-cert\") pod \"maas-controller-6d5898b76-x5w8m\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") " pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:33.083388 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:33.083335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvw56\" (UniqueName: \"kubernetes.io/projected/2efeab65-f83c-4ea6-ad99-03abb31788c9-kube-api-access-nvw56\") pod \"maas-controller-6d5898b76-x5w8m\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") " pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:33.085687 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:33.085665    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2efeab65-f83c-4ea6-ad99-03abb31788c9-webhook-cert\") pod \"maas-controller-6d5898b76-x5w8m\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") " pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:33.090867 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:33.090850    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvw56\" (UniqueName: \"kubernetes.io/projected/2efeab65-f83c-4ea6-ad99-03abb31788c9-kube-api-access-nvw56\") pod \"maas-controller-6d5898b76-x5w8m\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") " pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:33.174340 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:33.174318    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:33.302999 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:33.302974    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d5898b76-x5w8m"]
Jul 22 21:14:33.305418 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:14:33.305396    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efeab65_f83c_4ea6_ad99_03abb31788c9.slice/crio-3275ba059268b4f45bbbba960511548d3efd6d0aca2bb974126c69bc44b604b9 WatchSource:0}: Error finding container 3275ba059268b4f45bbbba960511548d3efd6d0aca2bb974126c69bc44b604b9: Status 404 returned error can't find the container with id 3275ba059268b4f45bbbba960511548d3efd6d0aca2bb974126c69bc44b604b9
Jul 22 21:14:33.306517 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:33.306500    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 21:14:34.291279 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:34.291250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-x5w8m" event={"ID":"2efeab65-f83c-4ea6-ad99-03abb31788c9","Type":"ContainerStarted","Data":"54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0"}
Jul 22 21:14:34.291662 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:34.291296    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-x5w8m" event={"ID":"2efeab65-f83c-4ea6-ad99-03abb31788c9","Type":"ContainerStarted","Data":"3275ba059268b4f45bbbba960511548d3efd6d0aca2bb974126c69bc44b604b9"}
Jul 22 21:14:34.291662 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:34.291371    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:34.310437 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:34.310373    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d5898b76-x5w8m" podStartSLOduration=1.788435346 podStartE2EDuration="2.310360025s" totalImagesPullingTime="521.924679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:14:34.309436607 +0000 UTC m=+1056.710238608" watchObservedRunningTime="2026-07-22 21:14:34.310360025 +0000 UTC m=+1056.711162009"
Jul 22 21:14:45.299090 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:45.299062    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:49.870013 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:49.869979    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:14:49.935688 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:49.935629    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d5898b76-x5w8m"]
Jul 22 21:14:49.935838 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:49.935799    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d5898b76-x5w8m" podUID="2efeab65-f83c-4ea6-ad99-03abb31788c9" containerName="manager" containerID="cri-o://54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0" gracePeriod=30
Jul 22 21:14:50.185310 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.185288    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:50.307732 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.307700    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvw56\" (UniqueName: \"kubernetes.io/projected/2efeab65-f83c-4ea6-ad99-03abb31788c9-kube-api-access-nvw56\") pod \"2efeab65-f83c-4ea6-ad99-03abb31788c9\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") "
Jul 22 21:14:50.307872 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.307739    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2efeab65-f83c-4ea6-ad99-03abb31788c9-webhook-cert\") pod \"2efeab65-f83c-4ea6-ad99-03abb31788c9\" (UID: \"2efeab65-f83c-4ea6-ad99-03abb31788c9\") "
Jul 22 21:14:50.309773 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.309743    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2efeab65-f83c-4ea6-ad99-03abb31788c9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2efeab65-f83c-4ea6-ad99-03abb31788c9" (UID: "2efeab65-f83c-4ea6-ad99-03abb31788c9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:14:50.309942 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.309753    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2efeab65-f83c-4ea6-ad99-03abb31788c9-kube-api-access-nvw56" (OuterVolumeSpecName: "kube-api-access-nvw56") pod "2efeab65-f83c-4ea6-ad99-03abb31788c9" (UID: "2efeab65-f83c-4ea6-ad99-03abb31788c9"). InnerVolumeSpecName "kube-api-access-nvw56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:14:50.366229 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.366199    2559 generic.go:356] "Generic (PLEG): container finished" podID="2efeab65-f83c-4ea6-ad99-03abb31788c9" containerID="54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0" exitCode=0
Jul 22 21:14:50.366348 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.366253    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-x5w8m"
Jul 22 21:14:50.366348 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.366299    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-x5w8m" event={"ID":"2efeab65-f83c-4ea6-ad99-03abb31788c9","Type":"ContainerDied","Data":"54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0"}
Jul 22 21:14:50.366467 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.366358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-x5w8m" event={"ID":"2efeab65-f83c-4ea6-ad99-03abb31788c9","Type":"ContainerDied","Data":"3275ba059268b4f45bbbba960511548d3efd6d0aca2bb974126c69bc44b604b9"}
Jul 22 21:14:50.366467 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.366390    2559 scope.go:117] "RemoveContainer" containerID="54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0"
Jul 22 21:14:50.374549 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.374529    2559 scope.go:117] "RemoveContainer" containerID="54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0"
Jul 22 21:14:50.374782 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:14:50.374765    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0\": container with ID starting with 54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0 not found: ID does not exist" containerID="54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0"
Jul 22 21:14:50.374841 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.374794    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0"} err="failed to get container status \"54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0\": rpc error: code = NotFound desc = could not find container \"54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0\": container with ID starting with 54de4b8ee2bbe668d7301ee348dc9927a39da370599ca8b63f866c97ee4ab5d0 not found: ID does not exist"
Jul 22 21:14:50.385172 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.385148    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d5898b76-x5w8m"]
Jul 22 21:14:50.388961 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.388942    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d5898b76-x5w8m"]
Jul 22 21:14:50.408547 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.408522    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvw56\" (UniqueName: \"kubernetes.io/projected/2efeab65-f83c-4ea6-ad99-03abb31788c9-kube-api-access-nvw56\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:14:50.408547 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:50.408546    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2efeab65-f83c-4ea6-ad99-03abb31788c9-webhook-cert\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:14:51.080296 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.080258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:14:51.186260 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.186227    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d5898b76-64r57"]
Jul 22 21:14:51.186737 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.186718    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2efeab65-f83c-4ea6-ad99-03abb31788c9" containerName="manager"
Jul 22 21:14:51.186802 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.186740    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2efeab65-f83c-4ea6-ad99-03abb31788c9" containerName="manager"
Jul 22 21:14:51.186839 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.186806    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2efeab65-f83c-4ea6-ad99-03abb31788c9" containerName="manager"
Jul 22 21:14:51.190989 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.190971    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.193088 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.193067    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 21:14:51.194010 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.193983    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9ggqk\""
Jul 22 21:14:51.194110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.194044    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 21:14:51.196731 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.196707    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d5898b76-64r57"]
Jul 22 21:14:51.315867 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.315841    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f88ll\" (UniqueName: \"kubernetes.io/projected/f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc-kube-api-access-f88ll\") pod \"maas-controller-6d5898b76-64r57\" (UID: \"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc\") " pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.315981 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.315893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc-webhook-cert\") pod \"maas-controller-6d5898b76-64r57\" (UID: \"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc\") " pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.417101 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.417075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc-webhook-cert\") pod \"maas-controller-6d5898b76-64r57\" (UID: \"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc\") " pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.417237 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.417133    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f88ll\" (UniqueName: \"kubernetes.io/projected/f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc-kube-api-access-f88ll\") pod \"maas-controller-6d5898b76-64r57\" (UID: \"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc\") " pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.419489 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.419467    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc-webhook-cert\") pod \"maas-controller-6d5898b76-64r57\" (UID: \"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc\") " pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.424783 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.424766    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f88ll\" (UniqueName: \"kubernetes.io/projected/f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc-kube-api-access-f88ll\") pod \"maas-controller-6d5898b76-64r57\" (UID: \"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc\") " pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.503456 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.503432    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:51.629374 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:51.629350    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d5898b76-64r57"]
Jul 22 21:14:51.631352 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:14:51.631326    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f7acab_dc3f_4540_bb5f_7745ee4c6fdc.slice/crio-103e0fc40d6fe7bc6462d736c76e4fd3df6cb984ecf6f4930bfc19cba8665d7a WatchSource:0}: Error finding container 103e0fc40d6fe7bc6462d736c76e4fd3df6cb984ecf6f4930bfc19cba8665d7a: Status 404 returned error can't find the container with id 103e0fc40d6fe7bc6462d736c76e4fd3df6cb984ecf6f4930bfc19cba8665d7a
Jul 22 21:14:52.077795 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:52.077768    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2efeab65-f83c-4ea6-ad99-03abb31788c9" path="/var/lib/kubelet/pods/2efeab65-f83c-4ea6-ad99-03abb31788c9/volumes"
Jul 22 21:14:52.378934 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:52.378842    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-64r57" event={"ID":"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc","Type":"ContainerStarted","Data":"52cc59818b93ee4cb8974655adc210cfb82f898f2ba047394b305c56a8f8bc94"}
Jul 22 21:14:52.378934 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:52.378896    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d5898b76-64r57" event={"ID":"f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc","Type":"ContainerStarted","Data":"103e0fc40d6fe7bc6462d736c76e4fd3df6cb984ecf6f4930bfc19cba8665d7a"}
Jul 22 21:14:52.378934 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:52.378932    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:14:52.395119 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:14:52.395075    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d5898b76-64r57" podStartSLOduration=1.057801957 podStartE2EDuration="1.395062418s" totalImagesPullingTime="337.260461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:14:52.393607186 +0000 UTC m=+1074.794409178" watchObservedRunningTime="2026-07-22 21:14:52.395062418 +0000 UTC m=+1074.795864408"
Jul 22 21:15:03.385168 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:03.385137    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d5898b76-64r57"
Jul 22 21:15:24.769879 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:24.769847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:24.923353 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:24.923314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:25.009423 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:25.009393    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:30.925688 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:30.925653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:31.038530 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:31.038496    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:31.175095 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:31.175065    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:31.281763 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:31.281692    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:37.018166 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:37.018134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:15:37.119701 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:15:37.119671    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:16:30.029409 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:16:30.029380    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:16:36.135297 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:16:36.135260    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:16:58.024504 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:16:58.024476    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:16:58.026761 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:16:58.026741    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:17:01.513080 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:17:01.513042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:17:01.576732 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:17:01.576707    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:18:38.872285 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:18:38.872253    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:18:41.194270 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:18:41.194240    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:19:32.135901 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:32.135866    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"]
Jul 22 21:19:32.136451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:32.136062    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl" podUID="502c2e33-3356-4b74-bcac-a2cbcc262b9a" containerName="manager" containerID="cri-o://3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6" gracePeriod=10
Jul 22 21:19:35.496879 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.496824    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:19:35.562887 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.562853    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/502c2e33-3356-4b74-bcac-a2cbcc262b9a-extensions-socket-volume\") pod \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") "
Jul 22 21:19:35.563062 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.562910    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prntj\" (UniqueName: \"kubernetes.io/projected/502c2e33-3356-4b74-bcac-a2cbcc262b9a-kube-api-access-prntj\") pod \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\" (UID: \"502c2e33-3356-4b74-bcac-a2cbcc262b9a\") "
Jul 22 21:19:35.563250 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.563226    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/502c2e33-3356-4b74-bcac-a2cbcc262b9a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "502c2e33-3356-4b74-bcac-a2cbcc262b9a" (UID: "502c2e33-3356-4b74-bcac-a2cbcc262b9a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:19:35.565504 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.565463    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/502c2e33-3356-4b74-bcac-a2cbcc262b9a-kube-api-access-prntj" (OuterVolumeSpecName: "kube-api-access-prntj") pod "502c2e33-3356-4b74-bcac-a2cbcc262b9a" (UID: "502c2e33-3356-4b74-bcac-a2cbcc262b9a"). InnerVolumeSpecName "kube-api-access-prntj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:19:35.664407 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.664368    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/502c2e33-3356-4b74-bcac-a2cbcc262b9a-extensions-socket-volume\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:19:35.664407 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.664405    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prntj\" (UniqueName: \"kubernetes.io/projected/502c2e33-3356-4b74-bcac-a2cbcc262b9a-kube-api-access-prntj\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:19:35.690279 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.690246    2559 generic.go:356] "Generic (PLEG): container finished" podID="502c2e33-3356-4b74-bcac-a2cbcc262b9a" containerID="3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6" exitCode=0
Jul 22 21:19:35.690430 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.690293    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"
Jul 22 21:19:35.690430 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.690328    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl" event={"ID":"502c2e33-3356-4b74-bcac-a2cbcc262b9a","Type":"ContainerDied","Data":"3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6"}
Jul 22 21:19:35.690430 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.690382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl" event={"ID":"502c2e33-3356-4b74-bcac-a2cbcc262b9a","Type":"ContainerDied","Data":"33b8418c72cbbbb689ec1de89af95892602aff9f328602c25052167925ee46eb"}
Jul 22 21:19:35.690430 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.690406    2559 scope.go:117] "RemoveContainer" containerID="3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6"
Jul 22 21:19:35.704108 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.704079    2559 scope.go:117] "RemoveContainer" containerID="3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6"
Jul 22 21:19:35.704530 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:19:35.704499    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6\": container with ID starting with 3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6 not found: ID does not exist" containerID="3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6"
Jul 22 21:19:35.704600 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.704541    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6"} err="failed to get container status \"3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6\": rpc error: code = NotFound desc = could not find container \"3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6\": container with ID starting with 3d1413910ed16a99c22251c376a7d0b4aef9eb61bed81ea85c619a90eb0fdac6 not found: ID does not exist"
Jul 22 21:19:35.761378 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.761297    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"]
Jul 22 21:19:35.805619 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:35.805587    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cwwbl"]
Jul 22 21:19:36.077584 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:19:36.077505    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="502c2e33-3356-4b74-bcac-a2cbcc262b9a" path="/var/lib/kubelet/pods/502c2e33-3356-4b74-bcac-a2cbcc262b9a/volumes"
Jul 22 21:20:33.194282 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.194250    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"]
Jul 22 21:20:33.194740 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.194584    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="502c2e33-3356-4b74-bcac-a2cbcc262b9a" containerName="manager"
Jul 22 21:20:33.194740 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.194595    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="502c2e33-3356-4b74-bcac-a2cbcc262b9a" containerName="manager"
Jul 22 21:20:33.194740 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.194661    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="502c2e33-3356-4b74-bcac-a2cbcc262b9a" containerName="manager"
Jul 22 21:20:33.197535 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.197519    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.202195 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.202158    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tj6rc\""
Jul 22 21:20:33.209130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.209108    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"]
Jul 22 21:20:33.300253 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.300219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cdcd6b2-d522-483f-b478-357cf6ed1e44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bdpjs\" (UID: \"5cdcd6b2-d522-483f-b478-357cf6ed1e44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.300379 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.300275    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vns9k\" (UniqueName: \"kubernetes.io/projected/5cdcd6b2-d522-483f-b478-357cf6ed1e44-kube-api-access-vns9k\") pod \"kuadrant-operator-controller-manager-86878c769f-bdpjs\" (UID: \"5cdcd6b2-d522-483f-b478-357cf6ed1e44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.400722 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.400695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cdcd6b2-d522-483f-b478-357cf6ed1e44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bdpjs\" (UID: \"5cdcd6b2-d522-483f-b478-357cf6ed1e44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.400830 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.400736    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vns9k\" (UniqueName: \"kubernetes.io/projected/5cdcd6b2-d522-483f-b478-357cf6ed1e44-kube-api-access-vns9k\") pod \"kuadrant-operator-controller-manager-86878c769f-bdpjs\" (UID: \"5cdcd6b2-d522-483f-b478-357cf6ed1e44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.401077 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.401058    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cdcd6b2-d522-483f-b478-357cf6ed1e44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bdpjs\" (UID: \"5cdcd6b2-d522-483f-b478-357cf6ed1e44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.409285 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.409267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vns9k\" (UniqueName: \"kubernetes.io/projected/5cdcd6b2-d522-483f-b478-357cf6ed1e44-kube-api-access-vns9k\") pod \"kuadrant-operator-controller-manager-86878c769f-bdpjs\" (UID: \"5cdcd6b2-d522-483f-b478-357cf6ed1e44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.507381 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.507328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.642260 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.640771    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"]
Jul 22 21:20:33.648412 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:20:33.648385    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cdcd6b2_d522_483f_b478_357cf6ed1e44.slice/crio-96e7405ec432dcc942c211d4b18c4858bba2c016fe4d2b85f7b5b00a3ea001f6 WatchSource:0}: Error finding container 96e7405ec432dcc942c211d4b18c4858bba2c016fe4d2b85f7b5b00a3ea001f6: Status 404 returned error can't find the container with id 96e7405ec432dcc942c211d4b18c4858bba2c016fe4d2b85f7b5b00a3ea001f6
Jul 22 21:20:33.950425 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.950388    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs" event={"ID":"5cdcd6b2-d522-483f-b478-357cf6ed1e44","Type":"ContainerStarted","Data":"bc417fa84bb18b60fe3c5be9c70b57b6b41eaa62e8ffbca0488bcdebf68a80e3"}
Jul 22 21:20:33.950583 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.950449    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs" event={"ID":"5cdcd6b2-d522-483f-b478-357cf6ed1e44","Type":"ContainerStarted","Data":"96e7405ec432dcc942c211d4b18c4858bba2c016fe4d2b85f7b5b00a3ea001f6"}
Jul 22 21:20:33.950583 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.950559    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:20:33.968046 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:33.967994    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs" podStartSLOduration=0.967981253 podStartE2EDuration="967.981253ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:20:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:20:33.966084327 +0000 UTC m=+1416.366886312" watchObservedRunningTime="2026-07-22 21:20:33.967981253 +0000 UTC m=+1416.368783236"
Jul 22 21:20:39.699790 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:39.699723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:20:44.958079 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:20:44.958049    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bdpjs"
Jul 22 21:21:30.136106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:21:30.136070    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:21:30.237080 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:21:30.237027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:21:58.052579 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:21:58.052552    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:21:58.055568 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:21:58.055548    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:23:27.754463 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:23:27.754429    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:23:37.622427 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:23:37.622394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:23:56.055844 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:23:56.055741    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:23:57.618890 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:23:57.618848    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:26:18.448237 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:26:18.448205    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:26:58.078797 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:26:58.078767    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:26:58.083106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:26:58.083086    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:27:07.851458 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:07.851425    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:10.878824 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:10.878788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:11.145771 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:11.145742    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:34.180631 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:34.180599    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:37.546494 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:37.546452    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:37.650385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:37.650357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:37.944347 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:37.944317    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:44.359806 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:44.359774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:44.470298 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:44.470265    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:44.691788 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:44.691752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:44.948732 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:44.948653    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:45.047989 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:45.047948    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:50.319567 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.319511    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:50.395162 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.395122    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"]
Jul 22 21:27:50.398684 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.398663    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.401329 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.401310    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d4a1f8ee-serving-cert\""
Jul 22 21:27:50.414467 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.414442    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"]
Jul 22 21:27:50.424134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.424111    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:50.480232 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.480171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.480382 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.480297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrfw2\" (UniqueName: \"kubernetes.io/projected/542772e1-a941-4efa-aded-f0d75021554a-kube-api-access-rrfw2\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.480382 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.480348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/542772e1-a941-4efa-aded-f0d75021554a-openshift-service-ca\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.517670 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.517639    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:50.580792 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.580714    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.580792 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.580781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrfw2\" (UniqueName: \"kubernetes.io/projected/542772e1-a941-4efa-aded-f0d75021554a-kube-api-access-rrfw2\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.580988 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.580820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/542772e1-a941-4efa-aded-f0d75021554a-openshift-service-ca\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.580988 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:27:50.580858    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-serving-cert: secret "maas-api-e2e-ait-d4a1f8ee-serving-cert" not found
Jul 22 21:27:50.580988 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:27:50.580938    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls podName:542772e1-a941-4efa-aded-f0d75021554a nodeName:}" failed. No retries permitted until 2026-07-22 21:27:51.080919399 +0000 UTC m=+1853.481721364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls") pod "maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p" (UID: "542772e1-a941-4efa-aded-f0d75021554a") : secret "maas-api-e2e-ait-d4a1f8ee-serving-cert" not found
Jul 22 21:27:50.581518 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.581496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/542772e1-a941-4efa-aded-f0d75021554a-openshift-service-ca\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.593698 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.593679    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrfw2\" (UniqueName: \"kubernetes.io/projected/542772e1-a941-4efa-aded-f0d75021554a-kube-api-access-rrfw2\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:50.707670 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:50.707642    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:51.053767 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.053735    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:51.084998 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.084976    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:51.087913 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.087891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls\") pod \"maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:51.158740 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.158714    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:27:51.308459 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.308389    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:51.451774 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.451751    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"]
Jul 22 21:27:51.454234 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:27:51.454203    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542772e1_a941_4efa_aded_f0d75021554a.slice/crio-c966a7c9cb88873dafc9ca8e630fc5e00065a83cef7f5e7cbd30395636321544 WatchSource:0}: Error finding container c966a7c9cb88873dafc9ca8e630fc5e00065a83cef7f5e7cbd30395636321544: Status 404 returned error can't find the container with id c966a7c9cb88873dafc9ca8e630fc5e00065a83cef7f5e7cbd30395636321544
Jul 22 21:27:51.455746 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.455726    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 21:27:51.922467 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:51.922435    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p" event={"ID":"542772e1-a941-4efa-aded-f0d75021554a","Type":"ContainerStarted","Data":"c966a7c9cb88873dafc9ca8e630fc5e00065a83cef7f5e7cbd30395636321544"}
Jul 22 21:27:52.929150 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:52.929117    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p" event={"ID":"542772e1-a941-4efa-aded-f0d75021554a","Type":"ContainerStarted","Data":"fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5"}
Jul 22 21:27:52.929532 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:52.929244    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:27:52.946921 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:52.946871    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p" podStartSLOduration=2.123040555 podStartE2EDuration="2.946856182s" totalImagesPullingTime="823.815627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:27:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:27:52.946357841 +0000 UTC m=+1855.347159827" watchObservedRunningTime="2026-07-22 21:27:52.946856182 +0000 UTC m=+1855.347658164"
Jul 22 21:27:58.940593 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:27:58.940560    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:28:26.293131 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.292950    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"]
Jul 22 21:28:26.293950 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.293897    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p" podUID="542772e1-a941-4efa-aded-f0d75021554a" containerName="maas-api" containerID="cri-o://fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5" gracePeriod=30
Jul 22 21:28:26.323072 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.323029    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:26.548272 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.548215    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:28:26.651969 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.651940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/542772e1-a941-4efa-aded-f0d75021554a-openshift-service-ca\") pod \"542772e1-a941-4efa-aded-f0d75021554a\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") "
Jul 22 21:28:26.652104 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.652030    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls\") pod \"542772e1-a941-4efa-aded-f0d75021554a\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") "
Jul 22 21:28:26.652104 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.652073    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrfw2\" (UniqueName: \"kubernetes.io/projected/542772e1-a941-4efa-aded-f0d75021554a-kube-api-access-rrfw2\") pod \"542772e1-a941-4efa-aded-f0d75021554a\" (UID: \"542772e1-a941-4efa-aded-f0d75021554a\") "
Jul 22 21:28:26.652399 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.652360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/542772e1-a941-4efa-aded-f0d75021554a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "542772e1-a941-4efa-aded-f0d75021554a" (UID: "542772e1-a941-4efa-aded-f0d75021554a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:28:26.654284 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.654256    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/542772e1-a941-4efa-aded-f0d75021554a-kube-api-access-rrfw2" (OuterVolumeSpecName: "kube-api-access-rrfw2") pod "542772e1-a941-4efa-aded-f0d75021554a" (UID: "542772e1-a941-4efa-aded-f0d75021554a"). InnerVolumeSpecName "kube-api-access-rrfw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:28:26.654364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.654262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "542772e1-a941-4efa-aded-f0d75021554a" (UID: "542772e1-a941-4efa-aded-f0d75021554a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:28:26.752795 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.752776    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/542772e1-a941-4efa-aded-f0d75021554a-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:28:26.752795 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.752796    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrfw2\" (UniqueName: \"kubernetes.io/projected/542772e1-a941-4efa-aded-f0d75021554a-kube-api-access-rrfw2\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:28:26.752906 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:26.752805    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/542772e1-a941-4efa-aded-f0d75021554a-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:28:27.081871 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.081842    2559 generic.go:356] "Generic (PLEG): container finished" podID="542772e1-a941-4efa-aded-f0d75021554a" containerID="fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5" exitCode=0
Jul 22 21:28:27.082029 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.081886    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"
Jul 22 21:28:27.082029 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.081890    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p" event={"ID":"542772e1-a941-4efa-aded-f0d75021554a","Type":"ContainerDied","Data":"fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5"}
Jul 22 21:28:27.082029 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.081991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p" event={"ID":"542772e1-a941-4efa-aded-f0d75021554a","Type":"ContainerDied","Data":"c966a7c9cb88873dafc9ca8e630fc5e00065a83cef7f5e7cbd30395636321544"}
Jul 22 21:28:27.082029 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.082011    2559 scope.go:117] "RemoveContainer" containerID="fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5"
Jul 22 21:28:27.093921 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.093723    2559 scope.go:117] "RemoveContainer" containerID="fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5"
Jul 22 21:28:27.094271 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:28:27.094242    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5\": container with ID starting with fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5 not found: ID does not exist" containerID="fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5"
Jul 22 21:28:27.094350 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.094281    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5"} err="failed to get container status \"fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5\": rpc error: code = NotFound desc = could not find container \"fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5\": container with ID starting with fdf87b8e4e7808cfac3d4fdc901e40abde566c30f6a2ebc864be6fdf273805b5 not found: ID does not exist"
Jul 22 21:28:27.107158 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.107135    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"]
Jul 22 21:28:27.111543 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:27.111523    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d4a1f8ee-6b8d486468-7xd9p"]
Jul 22 21:28:28.077928 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:28.077892    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="542772e1-a941-4efa-aded-f0d75021554a" path="/var/lib/kubelet/pods/542772e1-a941-4efa-aded-f0d75021554a/volumes"
Jul 22 21:28:30.943847 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:30.943816    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:31.099583 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:31.099548    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:31.329246 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:31.329149    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:31.583643 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:31.583561    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:31.682887 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:31.682850    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:31.815236 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:31.815209    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:32.151468 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:32.151432    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:32.402207 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:32.402102    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:34.611819 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.611787    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:34.689367 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.689330    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"]
Jul 22 21:28:34.689883 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.689868    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542772e1-a941-4efa-aded-f0d75021554a" containerName="maas-api"
Jul 22 21:28:34.689943 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.689887    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="542772e1-a941-4efa-aded-f0d75021554a" containerName="maas-api"
Jul 22 21:28:34.689988 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.689974    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="542772e1-a941-4efa-aded-f0d75021554a" containerName="maas-api"
Jul 22 21:28:34.693424 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.693403    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.695779 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.695758    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-9ab0a8ae-serving-cert\""
Jul 22 21:28:34.702233 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.702209    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"]
Jul 22 21:28:34.722112 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.722091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:34.804668 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.804637    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:34.818818 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.818794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7bee3-49a6-4611-a69e-aad0204564d4-openshift-service-ca\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.818919 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.818825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vzpv\" (UniqueName: \"kubernetes.io/projected/1db7bee3-49a6-4611-a69e-aad0204564d4-kube-api-access-8vzpv\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.818919 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.818845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db7bee3-49a6-4611-a69e-aad0204564d4-maas-api-tls\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.919484 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.919452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7bee3-49a6-4611-a69e-aad0204564d4-openshift-service-ca\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.919484 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.919482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vzpv\" (UniqueName: \"kubernetes.io/projected/1db7bee3-49a6-4611-a69e-aad0204564d4-kube-api-access-8vzpv\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.919667 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.919508    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db7bee3-49a6-4611-a69e-aad0204564d4-maas-api-tls\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.920092 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.920065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7bee3-49a6-4611-a69e-aad0204564d4-openshift-service-ca\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.922012 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.921990    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db7bee3-49a6-4611-a69e-aad0204564d4-maas-api-tls\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:34.926488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:34.926469    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vzpv\" (UniqueName: \"kubernetes.io/projected/1db7bee3-49a6-4611-a69e-aad0204564d4-kube-api-access-8vzpv\") pod \"maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:35.005871 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:35.005850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:35.138626 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:35.138600    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"]
Jul 22 21:28:35.141595 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:28:35.141563    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db7bee3_49a6_4611_a69e_aad0204564d4.slice/crio-52c15c0787603ec939823723f64bbd9458f2cbcb4df47f2aa32f152a0bef6d4d WatchSource:0}: Error finding container 52c15c0787603ec939823723f64bbd9458f2cbcb4df47f2aa32f152a0bef6d4d: Status 404 returned error can't find the container with id 52c15c0787603ec939823723f64bbd9458f2cbcb4df47f2aa32f152a0bef6d4d
Jul 22 21:28:35.214752 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:35.214679    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:35.316039 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:35.316015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:28:36.127117 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:36.127081    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq" event={"ID":"1db7bee3-49a6-4611-a69e-aad0204564d4","Type":"ContainerStarted","Data":"9dbfefc9b74f3ec6e44034caa8a0d0c5e06c3c31ebd38a39e683f51e430f713e"}
Jul 22 21:28:36.127481 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:36.127139    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq" event={"ID":"1db7bee3-49a6-4611-a69e-aad0204564d4","Type":"ContainerStarted","Data":"52c15c0787603ec939823723f64bbd9458f2cbcb4df47f2aa32f152a0bef6d4d"}
Jul 22 21:28:36.127481 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:36.127192    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:28:36.143304 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:36.143260    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq" podStartSLOduration=1.77896424 podStartE2EDuration="2.143243564s" totalImagesPullingTime="364.279324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:28:36.142229658 +0000 UTC m=+1898.543031662" watchObservedRunningTime="2026-07-22 21:28:36.143243564 +0000 UTC m=+1898.544045547"
Jul 22 21:28:42.136105 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:28:42.136075    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:29:03.662818 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:03.662764    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:05.485789 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:05.485757    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:10.030987 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.030396    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"]
Jul 22 21:29:10.031436 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.031270    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq" podUID="1db7bee3-49a6-4611-a69e-aad0204564d4" containerName="maas-api" containerID="cri-o://9dbfefc9b74f3ec6e44034caa8a0d0c5e06c3c31ebd38a39e683f51e430f713e" gracePeriod=30
Jul 22 21:29:10.055757 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.055736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:10.278140 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.278111    2559 generic.go:356] "Generic (PLEG): container finished" podID="1db7bee3-49a6-4611-a69e-aad0204564d4" containerID="9dbfefc9b74f3ec6e44034caa8a0d0c5e06c3c31ebd38a39e683f51e430f713e" exitCode=0
Jul 22 21:29:10.278270 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.278213    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq" event={"ID":"1db7bee3-49a6-4611-a69e-aad0204564d4","Type":"ContainerDied","Data":"9dbfefc9b74f3ec6e44034caa8a0d0c5e06c3c31ebd38a39e683f51e430f713e"}
Jul 22 21:29:10.278336 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.278279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq" event={"ID":"1db7bee3-49a6-4611-a69e-aad0204564d4","Type":"ContainerDied","Data":"52c15c0787603ec939823723f64bbd9458f2cbcb4df47f2aa32f152a0bef6d4d"}
Jul 22 21:29:10.278336 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.278302    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52c15c0787603ec939823723f64bbd9458f2cbcb4df47f2aa32f152a0bef6d4d"
Jul 22 21:29:10.287511 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.287466    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:29:10.387552 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.387510    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7bee3-49a6-4611-a69e-aad0204564d4-openshift-service-ca\") pod \"1db7bee3-49a6-4611-a69e-aad0204564d4\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") "
Jul 22 21:29:10.387714 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.387569    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vzpv\" (UniqueName: \"kubernetes.io/projected/1db7bee3-49a6-4611-a69e-aad0204564d4-kube-api-access-8vzpv\") pod \"1db7bee3-49a6-4611-a69e-aad0204564d4\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") "
Jul 22 21:29:10.387714 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.387594    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db7bee3-49a6-4611-a69e-aad0204564d4-maas-api-tls\") pod \"1db7bee3-49a6-4611-a69e-aad0204564d4\" (UID: \"1db7bee3-49a6-4611-a69e-aad0204564d4\") "
Jul 22 21:29:10.387944 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.387911    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1db7bee3-49a6-4611-a69e-aad0204564d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1db7bee3-49a6-4611-a69e-aad0204564d4" (UID: "1db7bee3-49a6-4611-a69e-aad0204564d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:29:10.389648 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.389623    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1db7bee3-49a6-4611-a69e-aad0204564d4-kube-api-access-8vzpv" (OuterVolumeSpecName: "kube-api-access-8vzpv") pod "1db7bee3-49a6-4611-a69e-aad0204564d4" (UID: "1db7bee3-49a6-4611-a69e-aad0204564d4"). InnerVolumeSpecName "kube-api-access-8vzpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:29:10.389767 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.389743    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1db7bee3-49a6-4611-a69e-aad0204564d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1db7bee3-49a6-4611-a69e-aad0204564d4" (UID: "1db7bee3-49a6-4611-a69e-aad0204564d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:29:10.488329 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.488284    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7bee3-49a6-4611-a69e-aad0204564d4-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:29:10.488329 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.488322    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vzpv\" (UniqueName: \"kubernetes.io/projected/1db7bee3-49a6-4611-a69e-aad0204564d4-kube-api-access-8vzpv\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:29:10.488329 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:10.488333    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db7bee3-49a6-4611-a69e-aad0204564d4-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:29:11.282506 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:11.282474    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"
Jul 22 21:29:11.302443 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:11.302417    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"]
Jul 22 21:29:11.306156 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:11.306134    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9ab0a8ae-67dfc5898b-zvkjq"]
Jul 22 21:29:12.076839 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:12.076808    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1db7bee3-49a6-4611-a69e-aad0204564d4" path="/var/lib/kubelet/pods/1db7bee3-49a6-4611-a69e-aad0204564d4/volumes"
Jul 22 21:29:14.695146 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:14.695109    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:14.844808 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:14.844777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:15.083062 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:15.082959    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:15.307084 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:15.307048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:15.628917 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:15.628886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:16.065211 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:16.065163    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:17.739433 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.739401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:17.835028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.834993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:17.908797 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.908764    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"]
Jul 22 21:29:17.909143 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.909129    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db7bee3-49a6-4611-a69e-aad0204564d4" containerName="maas-api"
Jul 22 21:29:17.909208 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.909146    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db7bee3-49a6-4611-a69e-aad0204564d4" containerName="maas-api"
Jul 22 21:29:17.909282 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.909269    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1db7bee3-49a6-4611-a69e-aad0204564d4" containerName="maas-api"
Jul 22 21:29:17.912517 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.912502    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:17.914792 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.914765    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1f4291a2-serving-cert\""
Jul 22 21:29:17.921169 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.921146    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"]
Jul 22 21:29:17.938136 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:17.938116    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:18.045998 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.045912    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebc15c4-dce6-4dbf-82f5-440acc303378-openshift-service-ca\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.045998 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.045948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.046340 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.045998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ppdr\" (UniqueName: \"kubernetes.io/projected/2ebc15c4-dce6-4dbf-82f5-440acc303378-kube-api-access-2ppdr\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.146882 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.146843    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebc15c4-dce6-4dbf-82f5-440acc303378-openshift-service-ca\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.147044 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.146893    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.147044 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.146954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ppdr\" (UniqueName: \"kubernetes.io/projected/2ebc15c4-dce6-4dbf-82f5-440acc303378-kube-api-access-2ppdr\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.147161 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:29:18.147041    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-serving-cert: secret "maas-api-e2e-ait-1f4291a2-serving-cert" not found
Jul 22 21:29:18.147161 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:29:18.147127    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls podName:2ebc15c4-dce6-4dbf-82f5-440acc303378 nodeName:}" failed. No retries permitted until 2026-07-22 21:29:18.647104481 +0000 UTC m=+1941.047906449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls") pod "maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt" (UID: "2ebc15c4-dce6-4dbf-82f5-440acc303378") : secret "maas-api-e2e-ait-1f4291a2-serving-cert" not found
Jul 22 21:29:18.147524 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.147507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebc15c4-dce6-4dbf-82f5-440acc303378-openshift-service-ca\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.155571 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.155548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ppdr\" (UniqueName: \"kubernetes.io/projected/2ebc15c4-dce6-4dbf-82f5-440acc303378-kube-api-access-2ppdr\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.304492 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.304413    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:18.407651 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.407622    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:18.510748 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.510719    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:18.650675 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.650641    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.653129 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.653101    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls\") pod \"maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:18.823564 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:18.823537    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:19.175856 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:19.175825    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"]
Jul 22 21:29:19.177535 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:29:19.177511    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ebc15c4_dce6_4dbf_82f5_440acc303378.slice/crio-5c7691151a50082ab500044771aae48734be50531daaf9de0ac1f80188e1de6f WatchSource:0}: Error finding container 5c7691151a50082ab500044771aae48734be50531daaf9de0ac1f80188e1de6f: Status 404 returned error can't find the container with id 5c7691151a50082ab500044771aae48734be50531daaf9de0ac1f80188e1de6f
Jul 22 21:29:19.318758 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:19.318727    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt" event={"ID":"2ebc15c4-dce6-4dbf-82f5-440acc303378","Type":"ContainerStarted","Data":"5c7691151a50082ab500044771aae48734be50531daaf9de0ac1f80188e1de6f"}
Jul 22 21:29:20.324595 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:20.324565    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt" event={"ID":"2ebc15c4-dce6-4dbf-82f5-440acc303378","Type":"ContainerStarted","Data":"e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4"}
Jul 22 21:29:20.324961 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:20.324617    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:20.345081 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:20.345040    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt" podStartSLOduration=2.883980582 podStartE2EDuration="3.345024948s" totalImagesPullingTime="461.044366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:29:20.342506032 +0000 UTC m=+1942.743308035" watchObservedRunningTime="2026-07-22 21:29:20.345024948 +0000 UTC m=+1942.745826939"
Jul 22 21:29:26.335284 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:26.335258    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:53.930245 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:53.930026    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"]
Jul 22 21:29:53.930843 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:53.930300    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt" podUID="2ebc15c4-dce6-4dbf-82f5-440acc303378" containerName="maas-api" containerID="cri-o://e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4" gracePeriod=30
Jul 22 21:29:53.951028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:53.951004    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:54.178981 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.178956    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:54.211953 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.211892    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebc15c4-dce6-4dbf-82f5-440acc303378-openshift-service-ca\") pod \"2ebc15c4-dce6-4dbf-82f5-440acc303378\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") "
Jul 22 21:29:54.211953 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.211919    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls\") pod \"2ebc15c4-dce6-4dbf-82f5-440acc303378\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") "
Jul 22 21:29:54.212114 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.211976    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ppdr\" (UniqueName: \"kubernetes.io/projected/2ebc15c4-dce6-4dbf-82f5-440acc303378-kube-api-access-2ppdr\") pod \"2ebc15c4-dce6-4dbf-82f5-440acc303378\" (UID: \"2ebc15c4-dce6-4dbf-82f5-440acc303378\") "
Jul 22 21:29:54.212255 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.212233    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ebc15c4-dce6-4dbf-82f5-440acc303378-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ebc15c4-dce6-4dbf-82f5-440acc303378" (UID: "2ebc15c4-dce6-4dbf-82f5-440acc303378"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:29:54.213984 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.213959    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ebc15c4-dce6-4dbf-82f5-440acc303378" (UID: "2ebc15c4-dce6-4dbf-82f5-440acc303378"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:29:54.214086 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.214004    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ebc15c4-dce6-4dbf-82f5-440acc303378-kube-api-access-2ppdr" (OuterVolumeSpecName: "kube-api-access-2ppdr") pod "2ebc15c4-dce6-4dbf-82f5-440acc303378" (UID: "2ebc15c4-dce6-4dbf-82f5-440acc303378"). InnerVolumeSpecName "kube-api-access-2ppdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:29:54.313554 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.313530    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ppdr\" (UniqueName: \"kubernetes.io/projected/2ebc15c4-dce6-4dbf-82f5-440acc303378-kube-api-access-2ppdr\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:29:54.313554 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.313553    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebc15c4-dce6-4dbf-82f5-440acc303378-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:29:54.313711 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.313563    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebc15c4-dce6-4dbf-82f5-440acc303378-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:29:54.475619 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.475556    2559 generic.go:356] "Generic (PLEG): container finished" podID="2ebc15c4-dce6-4dbf-82f5-440acc303378" containerID="e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4" exitCode=0
Jul 22 21:29:54.475619 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.475599    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"
Jul 22 21:29:54.475764 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.475640    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt" event={"ID":"2ebc15c4-dce6-4dbf-82f5-440acc303378","Type":"ContainerDied","Data":"e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4"}
Jul 22 21:29:54.475764 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.475700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt" event={"ID":"2ebc15c4-dce6-4dbf-82f5-440acc303378","Type":"ContainerDied","Data":"5c7691151a50082ab500044771aae48734be50531daaf9de0ac1f80188e1de6f"}
Jul 22 21:29:54.475764 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.475723    2559 scope.go:117] "RemoveContainer" containerID="e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4"
Jul 22 21:29:54.484474 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.484457    2559 scope.go:117] "RemoveContainer" containerID="e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4"
Jul 22 21:29:54.484733 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:29:54.484715    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4\": container with ID starting with e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4 not found: ID does not exist" containerID="e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4"
Jul 22 21:29:54.484779 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.484741    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4"} err="failed to get container status \"e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4\": rpc error: code = NotFound desc = could not find container \"e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4\": container with ID starting with e4be6ad6900a8d3dda4e51b7e2973a00faf64ad9bb775240037db27a2cce35b4 not found: ID does not exist"
Jul 22 21:29:54.495305 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.495285    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"]
Jul 22 21:29:54.498807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:54.498788    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f4291a2-84cd4d5f5d-lmtdt"]
Jul 22 21:29:56.077646 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:56.077613    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ebc15c4-dce6-4dbf-82f5-440acc303378" path="/var/lib/kubelet/pods/2ebc15c4-dce6-4dbf-82f5-440acc303378/volumes"
Jul 22 21:29:58.593785 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:58.593753    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:29:59.795918 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:29:59.795878    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:00.040252 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.040223    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:00.137322 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.137252    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg"]
Jul 22 21:30:00.137591 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.137580    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebc15c4-dce6-4dbf-82f5-440acc303378" containerName="maas-api"
Jul 22 21:30:00.137633 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.137592    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebc15c4-dce6-4dbf-82f5-440acc303378" containerName="maas-api"
Jul 22 21:30:00.137683 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.137674    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ebc15c4-dce6-4dbf-82f5-440acc303378" containerName="maas-api"
Jul 22 21:30:00.142250 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.142233    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg"
Jul 22 21:30:00.144844 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.144663    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz5vc\""
Jul 22 21:30:00.147301 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.147278    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg"]
Jul 22 21:30:00.155233 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.155054    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg"
Jul 22 21:30:00.263932 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.263902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:00.488855 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.488833    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg"]
Jul 22 21:30:00.491360 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:30:00.491322    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec19b5d_0abf_4281_94da_b5369091aa43.slice/crio-80d107579ea444063fe068cca267b1d6fdd3a030278ad436bfaab27b30c5b94e WatchSource:0}: Error finding container 80d107579ea444063fe068cca267b1d6fdd3a030278ad436bfaab27b30c5b94e: Status 404 returned error can't find the container with id 80d107579ea444063fe068cca267b1d6fdd3a030278ad436bfaab27b30c5b94e
Jul 22 21:30:00.504584 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.504559    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg" event={"ID":"5ec19b5d-0abf-4281-94da-b5369091aa43","Type":"ContainerStarted","Data":"80d107579ea444063fe068cca267b1d6fdd3a030278ad436bfaab27b30c5b94e"}
Jul 22 21:30:00.566359 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.566331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:00.852649 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:00.852574    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:02.998837 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:02.998804    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"]
Jul 22 21:30:03.003839 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.003788    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.006388 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.006365    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a5095382-serving-cert\""
Jul 22 21:30:03.015862 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.015805    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"]
Jul 22 21:30:03.050938 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.050910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:03.083731 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.083704    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-openshift-service-ca\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.083837 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.083745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-maas-api-tls\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.083878 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.083842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-558wr\" (UniqueName: \"kubernetes.io/projected/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-kube-api-access-558wr\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.151309 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.151268    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:03.184597 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.184565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-558wr\" (UniqueName: \"kubernetes.io/projected/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-kube-api-access-558wr\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.184697 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.184611    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-openshift-service-ca\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.184746 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.184721    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-maas-api-tls\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.185231 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.185210    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-openshift-service-ca\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.186935 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.186918    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-maas-api-tls\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.192172 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.192154    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-558wr\" (UniqueName: \"kubernetes.io/projected/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-kube-api-access-558wr\") pod \"maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.250160 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.250132    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:03.313113 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.313039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:03.451134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.451032    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"]
Jul 22 21:30:03.453445 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:30:03.453420    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267ed060_f5ac_4fd9_9a23_ffe1f22c69c7.slice/crio-af5f22b56c54e05a641561e8c0510e33f65e6863bcd4cc1c125bf63caa2aa3ac WatchSource:0}: Error finding container af5f22b56c54e05a641561e8c0510e33f65e6863bcd4cc1c125bf63caa2aa3ac: Status 404 returned error can't find the container with id af5f22b56c54e05a641561e8c0510e33f65e6863bcd4cc1c125bf63caa2aa3ac
Jul 22 21:30:03.523538 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.523505    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg" event={"ID":"5ec19b5d-0abf-4281-94da-b5369091aa43","Type":"ContainerStarted","Data":"47d6fdcdcbff169d40cb3128064d49702efb1c3698603c2869e66d3daa5e215f"}
Jul 22 21:30:03.524683 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.524662    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz" event={"ID":"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7","Type":"ContainerStarted","Data":"af5f22b56c54e05a641561e8c0510e33f65e6863bcd4cc1c125bf63caa2aa3ac"}
Jul 22 21:30:03.540933 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.540638    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg" podStartSLOduration=1.281895203 podStartE2EDuration="3.54061962s" totalImagesPullingTime="2.258724417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:30:03.539860919 +0000 UTC m=+1985.940662904" watchObservedRunningTime="2026-07-22 21:30:03.54061962 +0000 UTC m=+1985.941421619"
Jul 22 21:30:03.576685 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.576606    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:03.676215 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:03.676158    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:04.530309 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:04.530272    2559 generic.go:356] "Generic (PLEG): container finished" podID="5ec19b5d-0abf-4281-94da-b5369091aa43" containerID="47d6fdcdcbff169d40cb3128064d49702efb1c3698603c2869e66d3daa5e215f" exitCode=0
Jul 22 21:30:04.530735 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:04.530365    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg" event={"ID":"5ec19b5d-0abf-4281-94da-b5369091aa43","Type":"ContainerDied","Data":"47d6fdcdcbff169d40cb3128064d49702efb1c3698603c2869e66d3daa5e215f"}
Jul 22 21:30:04.531872 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:04.531848    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz" event={"ID":"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7","Type":"ContainerStarted","Data":"43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887"}
Jul 22 21:30:04.531969 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:04.531893    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:04.585335 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:04.585290    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz" podStartSLOduration=1.956358815 podStartE2EDuration="2.58527588s" totalImagesPullingTime="628.917065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:30:04.583810264 +0000 UTC m=+1986.984612271" watchObservedRunningTime="2026-07-22 21:30:04.58527588 +0000 UTC m=+1986.986077892"
Jul 22 21:30:05.676157 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:05.676134    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg"
Jul 22 21:30:06.541940 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:06.541915    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg"
Jul 22 21:30:06.542110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:06.541910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745930-4phlg" event={"ID":"5ec19b5d-0abf-4281-94da-b5369091aa43","Type":"ContainerDied","Data":"80d107579ea444063fe068cca267b1d6fdd3a030278ad436bfaab27b30c5b94e"}
Jul 22 21:30:06.542110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:06.542032    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80d107579ea444063fe068cca267b1d6fdd3a030278ad436bfaab27b30c5b94e"
Jul 22 21:30:10.541944 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:10.541913    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:11.587047 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:11.587006    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:11.686249 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:11.686220    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:24.354287 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:24.354241    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:24.453565 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:24.453533    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:38.221367 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.221333    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"]
Jul 22 21:30:38.221923 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.221567    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz" podUID="267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" containerName="maas-api" containerID="cri-o://43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887" gracePeriod=30
Jul 22 21:30:38.242743 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.242714    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:38.511131 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.511110    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:38.652930 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.652899    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-openshift-service-ca\") pod \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") "
Jul 22 21:30:38.652930 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.652931    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-558wr\" (UniqueName: \"kubernetes.io/projected/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-kube-api-access-558wr\") pod \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") "
Jul 22 21:30:38.653110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.652971    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-maas-api-tls\") pod \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\" (UID: \"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7\") "
Jul 22 21:30:38.653238 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.653217    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" (UID: "267ed060-f5ac-4fd9-9a23-ffe1f22c69c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:30:38.655010 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.654978    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" (UID: "267ed060-f5ac-4fd9-9a23-ffe1f22c69c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:30:38.655111 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.655064    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-kube-api-access-558wr" (OuterVolumeSpecName: "kube-api-access-558wr") pod "267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" (UID: "267ed060-f5ac-4fd9-9a23-ffe1f22c69c7"). InnerVolumeSpecName "kube-api-access-558wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:30:38.696399 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.696371    2559 generic.go:356] "Generic (PLEG): container finished" podID="267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" containerID="43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887" exitCode=0
Jul 22 21:30:38.696498 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.696424    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"
Jul 22 21:30:38.696498 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.696460    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz" event={"ID":"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7","Type":"ContainerDied","Data":"43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887"}
Jul 22 21:30:38.696621 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.696521    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz" event={"ID":"267ed060-f5ac-4fd9-9a23-ffe1f22c69c7","Type":"ContainerDied","Data":"af5f22b56c54e05a641561e8c0510e33f65e6863bcd4cc1c125bf63caa2aa3ac"}
Jul 22 21:30:38.696621 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.696558    2559 scope.go:117] "RemoveContainer" containerID="43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887"
Jul 22 21:30:38.705775 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.705758    2559 scope.go:117] "RemoveContainer" containerID="43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887"
Jul 22 21:30:38.706018 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:30:38.706000    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887\": container with ID starting with 43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887 not found: ID does not exist" containerID="43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887"
Jul 22 21:30:38.706070 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.706027    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887"} err="failed to get container status \"43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887\": rpc error: code = NotFound desc = could not find container \"43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887\": container with ID starting with 43e489b439da0b302804c96cb2cf8fba178fa1187fb461a6a5113aaf606ad887 not found: ID does not exist"
Jul 22 21:30:38.716414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.716370    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"]
Jul 22 21:30:38.718609 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.718588    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a5095382-6b9cd7dfc6-972vz"]
Jul 22 21:30:38.754215 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.754161    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:30:38.754215 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.754195    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:30:38.754215 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:38.754205    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-558wr\" (UniqueName: \"kubernetes.io/projected/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7-kube-api-access-558wr\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:30:40.078014 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:40.077975    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" path="/var/lib/kubelet/pods/267ed060-f5ac-4fd9-9a23-ffe1f22c69c7/volumes"
Jul 22 21:30:42.883989 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:42.883952    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:43.047047 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:43.047011    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:43.297633 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:43.297601    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:43.781214 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:43.781160    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:43.882384 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:43.882357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:43.982736 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:43.982708    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:45.398141 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:45.398107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:51.024813 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:51.024774    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:51.131445 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:51.131393    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:51.225552 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:51.225499    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:51.325154 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:51.325086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:30:51.702420 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:30:51.702396    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:17.673892 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:17.673861    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:24.178959 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:24.178872    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:24.282801 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:24.282768    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:24.379334 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:24.379297    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:40.276094 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:40.276059    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:46.161000 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:46.160964    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:46.275158 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:46.275125    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:46.362227 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:46.362198    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:56.891770 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:56.891722    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:57.719292 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:57.719258    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:31:58.106659 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:58.106570    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:31:58.114203 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:31:58.114161    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:32:03.849054 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:03.849014    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:03.956464 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:03.956434    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:04.068504 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:04.068430    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:04.184560 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:04.184527    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:04.903872 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:04.903839    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:05.003985 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:05.003949    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:05.110953 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:05.110926    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:06.755734 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:06.755701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:13.237099 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:13.237066    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:13.337103 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:13.337077    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:14.024968 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:14.024930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:19.931131 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:19.931092    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:20.032927 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:20.032892    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:20.162906 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:20.162874    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:20.515536 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:20.515482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:20.752799 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:20.752764    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:23.378500 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.378440    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:23.421838 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.421808    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"]
Jul 22 21:32:23.422363 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.422345    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" containerName="maas-api"
Jul 22 21:32:23.422457 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.422366    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" containerName="maas-api"
Jul 22 21:32:23.422457 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.422387    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ec19b5d-0abf-4281-94da-b5369091aa43" containerName="cleanup"
Jul 22 21:32:23.422457 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.422396    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ec19b5d-0abf-4281-94da-b5369091aa43" containerName="cleanup"
Jul 22 21:32:23.422555 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.422490    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="267ed060-f5ac-4fd9-9a23-ffe1f22c69c7" containerName="maas-api"
Jul 22 21:32:23.422555 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.422501    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ec19b5d-0abf-4281-94da-b5369091aa43" containerName="cleanup"
Jul 22 21:32:23.425720 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.425703    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.427950 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.427932    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-84ba5c64-serving-cert\""
Jul 22 21:32:23.432884 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.432863    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"]
Jul 22 21:32:23.449282 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.449259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.449390 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.449306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a805b83a-fba5-404c-86e1-d07a5bfb0947-openshift-service-ca\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.449390 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.449336    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcb98\" (UniqueName: \"kubernetes.io/projected/a805b83a-fba5-404c-86e1-d07a5bfb0947-kube-api-access-xcb98\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.491985 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.491956    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:23.550240 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.550202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.550381 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.550264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a805b83a-fba5-404c-86e1-d07a5bfb0947-openshift-service-ca\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.550381 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.550300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcb98\" (UniqueName: \"kubernetes.io/projected/a805b83a-fba5-404c-86e1-d07a5bfb0947-kube-api-access-xcb98\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.550490 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:32:23.550381    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-serving-cert: secret "maas-api-e2e-mt-84ba5c64-serving-cert" not found
Jul 22 21:32:23.550490 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:32:23.550458    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls podName:a805b83a-fba5-404c-86e1-d07a5bfb0947 nodeName:}" failed. No retries permitted until 2026-07-22 21:32:24.0504383 +0000 UTC m=+2126.451240279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls") pod "maas-api-e2e-mt-84ba5c64-787b99cddb-r897g" (UID: "a805b83a-fba5-404c-86e1-d07a5bfb0947") : secret "maas-api-e2e-mt-84ba5c64-serving-cert" not found
Jul 22 21:32:23.551084 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.551063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a805b83a-fba5-404c-86e1-d07a5bfb0947-openshift-service-ca\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.559364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.559345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcb98\" (UniqueName: \"kubernetes.io/projected/a805b83a-fba5-404c-86e1-d07a5bfb0947-kube-api-access-xcb98\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:23.579238 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.579172    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:23.733619 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:23.733589    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:24.021115 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:24.020982    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:24.054683 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:24.054642    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:24.057022 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:24.057001    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls\") pod \"maas-api-e2e-mt-84ba5c64-787b99cddb-r897g\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:24.123986 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:24.123957    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:24.338894 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:24.338783    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:24.490253 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:24.490226    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"]
Jul 22 21:32:24.492368 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:32:24.492344    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda805b83a_fba5_404c_86e1_d07a5bfb0947.slice/crio-d34386777e790d96c61d2e4cba97684438e1854949c409d3cd2b45e91d0181c7 WatchSource:0}: Error finding container d34386777e790d96c61d2e4cba97684438e1854949c409d3cd2b45e91d0181c7: Status 404 returned error can't find the container with id d34386777e790d96c61d2e4cba97684438e1854949c409d3cd2b45e91d0181c7
Jul 22 21:32:25.195578 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:25.195539    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g" event={"ID":"a805b83a-fba5-404c-86e1-d07a5bfb0947","Type":"ContainerStarted","Data":"d34386777e790d96c61d2e4cba97684438e1854949c409d3cd2b45e91d0181c7"}
Jul 22 21:32:26.201930 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:26.201894    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g" event={"ID":"a805b83a-fba5-404c-86e1-d07a5bfb0947","Type":"ContainerStarted","Data":"f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613"}
Jul 22 21:32:26.202381 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:26.201967    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:26.220624 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:26.220578    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g" podStartSLOduration=2.445541963 podStartE2EDuration="3.220562971s" totalImagesPullingTime="775.021008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:32:26.219088721 +0000 UTC m=+2128.619890704" watchObservedRunningTime="2026-07-22 21:32:26.220562971 +0000 UTC m=+2128.621364953"
Jul 22 21:32:27.748794 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:27.748759    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:28.397089 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:28.397054    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:28.494391 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:28.494361    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:28.842051 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:28.841971    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:32.215403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:32.215376    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:34.638133 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:34.638103    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:34.736782 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:34.736752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:45.295322 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.295287    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"]
Jul 22 21:32:45.296596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.296558    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g" podUID="a805b83a-fba5-404c-86e1-d07a5bfb0947" containerName="maas-api" containerID="cri-o://f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613" gracePeriod=30
Jul 22 21:32:45.324038 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.324009    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:45.548356 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.548292    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:45.630452 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.630421    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a805b83a-fba5-404c-86e1-d07a5bfb0947-openshift-service-ca\") pod \"a805b83a-fba5-404c-86e1-d07a5bfb0947\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") "
Jul 22 21:32:45.630452 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.630455    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcb98\" (UniqueName: \"kubernetes.io/projected/a805b83a-fba5-404c-86e1-d07a5bfb0947-kube-api-access-xcb98\") pod \"a805b83a-fba5-404c-86e1-d07a5bfb0947\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") "
Jul 22 21:32:45.630631 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.630501    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls\") pod \"a805b83a-fba5-404c-86e1-d07a5bfb0947\" (UID: \"a805b83a-fba5-404c-86e1-d07a5bfb0947\") "
Jul 22 21:32:45.630803 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.630778    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a805b83a-fba5-404c-86e1-d07a5bfb0947-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a805b83a-fba5-404c-86e1-d07a5bfb0947" (UID: "a805b83a-fba5-404c-86e1-d07a5bfb0947"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:32:45.632473 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.632445    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a805b83a-fba5-404c-86e1-d07a5bfb0947-kube-api-access-xcb98" (OuterVolumeSpecName: "kube-api-access-xcb98") pod "a805b83a-fba5-404c-86e1-d07a5bfb0947" (UID: "a805b83a-fba5-404c-86e1-d07a5bfb0947"). InnerVolumeSpecName "kube-api-access-xcb98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:32:45.632575 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.632475    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a805b83a-fba5-404c-86e1-d07a5bfb0947" (UID: "a805b83a-fba5-404c-86e1-d07a5bfb0947"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:32:45.731114 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.731087    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a805b83a-fba5-404c-86e1-d07a5bfb0947-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:32:45.731114 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.731112    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcb98\" (UniqueName: \"kubernetes.io/projected/a805b83a-fba5-404c-86e1-d07a5bfb0947-kube-api-access-xcb98\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:32:45.731282 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:45.731122    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a805b83a-fba5-404c-86e1-d07a5bfb0947-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:32:46.306451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.306412    2559 generic.go:356] "Generic (PLEG): container finished" podID="a805b83a-fba5-404c-86e1-d07a5bfb0947" containerID="f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613" exitCode=0
Jul 22 21:32:46.306831 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.306466    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"
Jul 22 21:32:46.306831 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.306497    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g" event={"ID":"a805b83a-fba5-404c-86e1-d07a5bfb0947","Type":"ContainerDied","Data":"f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613"}
Jul 22 21:32:46.306831 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.306550    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g" event={"ID":"a805b83a-fba5-404c-86e1-d07a5bfb0947","Type":"ContainerDied","Data":"d34386777e790d96c61d2e4cba97684438e1854949c409d3cd2b45e91d0181c7"}
Jul 22 21:32:46.306831 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.306572    2559 scope.go:117] "RemoveContainer" containerID="f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613"
Jul 22 21:32:46.314997 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.314979    2559 scope.go:117] "RemoveContainer" containerID="f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613"
Jul 22 21:32:46.315267 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:32:46.315248    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613\": container with ID starting with f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613 not found: ID does not exist" containerID="f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613"
Jul 22 21:32:46.315337 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.315277    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613"} err="failed to get container status \"f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613\": rpc error: code = NotFound desc = could not find container \"f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613\": container with ID starting with f1ba160c59c932fd7e36d87bf25e053b0e09a1810f7592fa77b55f5145370613 not found: ID does not exist"
Jul 22 21:32:46.322602 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.322580    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"]
Jul 22 21:32:46.329492 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:46.329469    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84ba5c64-787b99cddb-r897g"]
Jul 22 21:32:48.077374 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:48.077345    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a805b83a-fba5-404c-86e1-d07a5bfb0947" path="/var/lib/kubelet/pods/a805b83a-fba5-404c-86e1-d07a5bfb0947/volumes"
Jul 22 21:32:49.982836 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:49.982758    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:50.371566 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:50.371482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:56.226450 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:56.226395    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:56.392241 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:56.392208    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:56.552363 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:56.552265    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:56.787141 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:56.787097    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:56.925090 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:56.925060    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:57.918862 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:57.918806    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:32:58.019704 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:32:58.019654    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:02.794975 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:02.794939    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:02.903827 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:02.903794    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:02.996566 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:02.996539    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:03.162015 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:03.161984    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:03.368894 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:03.368864    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:08.867946 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:08.867910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:08.970732 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:08.970705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:15.693651 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:15.693621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:16.002071 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:16.001995    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:16.167312 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:16.167271    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:19.341225 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.341170    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:19.440511 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.440465    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:19.607973 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.607899    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"]
Jul 22 21:33:19.608334 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.608316    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a805b83a-fba5-404c-86e1-d07a5bfb0947" containerName="maas-api"
Jul 22 21:33:19.608417 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.608337    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a805b83a-fba5-404c-86e1-d07a5bfb0947" containerName="maas-api"
Jul 22 21:33:19.608502 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.608482    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a805b83a-fba5-404c-86e1-d07a5bfb0947" containerName="maas-api"
Jul 22 21:33:19.610540 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.610524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.612660 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.612641    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e1e05235-serving-cert\""
Jul 22 21:33:19.622137 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.622111    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"]
Jul 22 21:33:19.649526 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.649502    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:19.721021 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.720996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:19.789824 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.789801    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48838a6a-6627-445f-9873-26f8867f5a55-openshift-service-ca\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.789932 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.789850    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48838a6a-6627-445f-9873-26f8867f5a55-maas-api-tls\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.789975 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.789939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6m9r\" (UniqueName: \"kubernetes.io/projected/48838a6a-6627-445f-9873-26f8867f5a55-kube-api-access-f6m9r\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.891315 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.891289    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48838a6a-6627-445f-9873-26f8867f5a55-maas-api-tls\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.891420 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.891343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6m9r\" (UniqueName: \"kubernetes.io/projected/48838a6a-6627-445f-9873-26f8867f5a55-kube-api-access-f6m9r\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.891420 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.891372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48838a6a-6627-445f-9873-26f8867f5a55-openshift-service-ca\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.892000 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.891970    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48838a6a-6627-445f-9873-26f8867f5a55-openshift-service-ca\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.893696 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.893677    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48838a6a-6627-445f-9873-26f8867f5a55-maas-api-tls\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.899288 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.899267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6m9r\" (UniqueName: \"kubernetes.io/projected/48838a6a-6627-445f-9873-26f8867f5a55-kube-api-access-f6m9r\") pod \"maas-api-e2e-mt-e1e05235-94b45cc57-mxthz\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:19.920261 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:19.920239    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:20.059855 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:20.059831    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"]
Jul 22 21:33:20.061790 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:33:20.061764    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48838a6a_6627_445f_9873_26f8867f5a55.slice/crio-eabee4af990238503eca1700a46af611c80091fb28f4adc95ef729617adb2d2d WatchSource:0}: Error finding container eabee4af990238503eca1700a46af611c80091fb28f4adc95ef729617adb2d2d: Status 404 returned error can't find the container with id eabee4af990238503eca1700a46af611c80091fb28f4adc95ef729617adb2d2d
Jul 22 21:33:20.063111 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:20.063094    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 21:33:20.162381 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:20.162320    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:20.260709 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:20.260684    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:20.464150 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:20.464121    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz" event={"ID":"48838a6a-6627-445f-9873-26f8867f5a55","Type":"ContainerStarted","Data":"eabee4af990238503eca1700a46af611c80091fb28f4adc95ef729617adb2d2d"}
Jul 22 21:33:21.470000 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:21.469966    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz" event={"ID":"48838a6a-6627-445f-9873-26f8867f5a55","Type":"ContainerStarted","Data":"7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593"}
Jul 22 21:33:21.470419 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:21.470079    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:21.487946 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:21.487903    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz" podStartSLOduration=2.097211272 podStartE2EDuration="2.487889358s" totalImagesPullingTime="390.678086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:33:21.486431666 +0000 UTC m=+2183.887233652" watchObservedRunningTime="2026-07-22 21:33:21.487889358 +0000 UTC m=+2183.888691341"
Jul 22 21:33:23.859672 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:23.859641    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:27.479986 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:27.479960    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:33:30.132022 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:30.131988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:33:57.201762 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.201730    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"]
Jul 22 21:33:57.205351 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.205335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"
Jul 22 21:33:57.207493 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.207473    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz5vc\""
Jul 22 21:33:57.211195 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.211154    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"]
Jul 22 21:33:57.274602 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.274571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72444975-d195-4e58-98f4-007979d764b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf\" (UID: \"72444975-d195-4e58-98f4-007979d764b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"
Jul 22 21:33:57.375534 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.375503    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72444975-d195-4e58-98f4-007979d764b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf\" (UID: \"72444975-d195-4e58-98f4-007979d764b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"
Jul 22 21:33:57.376059 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.376042    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72444975-d195-4e58-98f4-007979d764b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf\" (UID: \"72444975-d195-4e58-98f4-007979d764b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"
Jul 22 21:33:57.518130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.518061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"
Jul 22 21:33:57.655711 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:57.655681    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"]
Jul 22 21:33:57.657104 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:33:57.657068    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72444975_d195_4e58_98f4_007979d764b9.slice/crio-2b4bad5c742470357e5a324d3c1faa24bccf17ddc795ae33051e8b5399d6cdc8 WatchSource:0}: Error finding container 2b4bad5c742470357e5a324d3c1faa24bccf17ddc795ae33051e8b5399d6cdc8: Status 404 returned error can't find the container with id 2b4bad5c742470357e5a324d3c1faa24bccf17ddc795ae33051e8b5399d6cdc8
Jul 22 21:33:58.651951 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:58.651916    2559 generic.go:356] "Generic (PLEG): container finished" podID="72444975-d195-4e58-98f4-007979d764b9" containerID="732249a51f0a87bf7badb4f1fbeff6f969a9e372bd62b7459972e90d836fa349" exitCode=0
Jul 22 21:33:58.652317 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:58.652008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf" event={"ID":"72444975-d195-4e58-98f4-007979d764b9","Type":"ContainerDied","Data":"732249a51f0a87bf7badb4f1fbeff6f969a9e372bd62b7459972e90d836fa349"}
Jul 22 21:33:58.652317 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:58.652076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf" event={"ID":"72444975-d195-4e58-98f4-007979d764b9","Type":"ContainerStarted","Data":"2b4bad5c742470357e5a324d3c1faa24bccf17ddc795ae33051e8b5399d6cdc8"}
Jul 22 21:33:59.788923 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:59.788902    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"
Jul 22 21:33:59.895812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:59.895778    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72444975-d195-4e58-98f4-007979d764b9-maas-api-service-ca\") pod \"72444975-d195-4e58-98f4-007979d764b9\" (UID: \"72444975-d195-4e58-98f4-007979d764b9\") "
Jul 22 21:33:59.896115 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:59.896092    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72444975-d195-4e58-98f4-007979d764b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72444975-d195-4e58-98f4-007979d764b9" (UID: "72444975-d195-4e58-98f4-007979d764b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:33:59.996799 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:33:59.996733    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72444975-d195-4e58-98f4-007979d764b9-maas-api-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:34:00.663446 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:00.663417    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"
Jul 22 21:34:00.663620 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:00.663452    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf" event={"ID":"72444975-d195-4e58-98f4-007979d764b9","Type":"ContainerDied","Data":"2b4bad5c742470357e5a324d3c1faa24bccf17ddc795ae33051e8b5399d6cdc8"}
Jul 22 21:34:00.663620 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:00.663506    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b4bad5c742470357e5a324d3c1faa24bccf17ddc795ae33051e8b5399d6cdc8"
Jul 22 21:34:12.564477 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:12.564446    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:12.631510 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:12.631474    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"]
Jul 22 21:34:12.631766 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:12.631717    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz" podUID="48838a6a-6627-445f-9873-26f8867f5a55" containerName="maas-api" containerID="cri-o://7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593" gracePeriod=30
Jul 22 21:34:12.879039 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:12.879016    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:34:13.002710 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.002684    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48838a6a-6627-445f-9873-26f8867f5a55-openshift-service-ca\") pod \"48838a6a-6627-445f-9873-26f8867f5a55\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") "
Jul 22 21:34:13.002852 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.002754    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6m9r\" (UniqueName: \"kubernetes.io/projected/48838a6a-6627-445f-9873-26f8867f5a55-kube-api-access-f6m9r\") pod \"48838a6a-6627-445f-9873-26f8867f5a55\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") "
Jul 22 21:34:13.002852 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.002773    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48838a6a-6627-445f-9873-26f8867f5a55-maas-api-tls\") pod \"48838a6a-6627-445f-9873-26f8867f5a55\" (UID: \"48838a6a-6627-445f-9873-26f8867f5a55\") "
Jul 22 21:34:13.003059 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.003035    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48838a6a-6627-445f-9873-26f8867f5a55-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48838a6a-6627-445f-9873-26f8867f5a55" (UID: "48838a6a-6627-445f-9873-26f8867f5a55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:34:13.004742 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.004719    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48838a6a-6627-445f-9873-26f8867f5a55-kube-api-access-f6m9r" (OuterVolumeSpecName: "kube-api-access-f6m9r") pod "48838a6a-6627-445f-9873-26f8867f5a55" (UID: "48838a6a-6627-445f-9873-26f8867f5a55"). InnerVolumeSpecName "kube-api-access-f6m9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:34:13.004804 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.004742    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48838a6a-6627-445f-9873-26f8867f5a55-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48838a6a-6627-445f-9873-26f8867f5a55" (UID: "48838a6a-6627-445f-9873-26f8867f5a55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:34:13.103956 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.103931    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6m9r\" (UniqueName: \"kubernetes.io/projected/48838a6a-6627-445f-9873-26f8867f5a55-kube-api-access-f6m9r\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:34:13.103956 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.103952    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48838a6a-6627-445f-9873-26f8867f5a55-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:34:13.104103 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.103961    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48838a6a-6627-445f-9873-26f8867f5a55-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:34:13.720765 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.720721    2559 generic.go:356] "Generic (PLEG): container finished" podID="48838a6a-6627-445f-9873-26f8867f5a55" containerID="7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593" exitCode=0
Jul 22 21:34:13.721248 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.720777    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"
Jul 22 21:34:13.721248 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.720805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz" event={"ID":"48838a6a-6627-445f-9873-26f8867f5a55","Type":"ContainerDied","Data":"7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593"}
Jul 22 21:34:13.721248 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.720880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz" event={"ID":"48838a6a-6627-445f-9873-26f8867f5a55","Type":"ContainerDied","Data":"eabee4af990238503eca1700a46af611c80091fb28f4adc95ef729617adb2d2d"}
Jul 22 21:34:13.721248 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.720903    2559 scope.go:117] "RemoveContainer" containerID="7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593"
Jul 22 21:34:13.730343 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.730326    2559 scope.go:117] "RemoveContainer" containerID="7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593"
Jul 22 21:34:13.730582 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:34:13.730564    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593\": container with ID starting with 7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593 not found: ID does not exist" containerID="7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593"
Jul 22 21:34:13.730623 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.730590    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593"} err="failed to get container status \"7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593\": rpc error: code = NotFound desc = could not find container \"7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593\": container with ID starting with 7af7f4905a3fd28e7a318d639c5f88e45dda3b9b806ca2b3aa144044f25d9593 not found: ID does not exist"
Jul 22 21:34:13.741797 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.741769    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"]
Jul 22 21:34:13.747982 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:13.747964    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e1e05235-94b45cc57-mxthz"]
Jul 22 21:34:14.008996 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:14.008902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:14.077111 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:14.077082    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48838a6a-6627-445f-9873-26f8867f5a55" path="/var/lib/kubelet/pods/48838a6a-6627-445f-9873-26f8867f5a55/volumes"
Jul 22 21:34:17.297947 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:17.297913    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"]
Jul 22 21:34:17.300694 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:17.300666    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e1e05235-zrpmf"]
Jul 22 21:34:17.301038 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:17.301021    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:18.077452 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:18.077424    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72444975-d195-4e58-98f4-007979d764b9" path="/var/lib/kubelet/pods/72444975-d195-4e58-98f4-007979d764b9/volumes"
Jul 22 21:34:19.521902 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:19.521866    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:24.598984 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:24.598896    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:41.512695 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:41.512661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:45.232464 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:45.232432    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:45.296713 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:45.296687    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:34:58.176548 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:34:58.176514    2559 scope.go:117] "RemoveContainer" containerID="9dbfefc9b74f3ec6e44034caa8a0d0c5e06c3c31ebd38a39e683f51e430f713e"
Jul 22 21:35:01.611237 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:01.611196    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:02.022480 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:02.022453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:07.551445 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:07.551413    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:07.739016 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:07.738988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:07.825426 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:07.825342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:08.055365 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:08.055331    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:09.178610 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:09.178581    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:09.301231 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:09.301203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:09.417429 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:09.417397    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:10.160978 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:10.160948    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:10.260472 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:10.260442    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:10.360577 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:10.360536    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:11.775684 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:11.775646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:12.355206 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:12.355130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:12.455239 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:12.455202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:17.468542 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:17.468503    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:17.567251 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:17.567224    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:18.136426 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:18.136395    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:23.996618 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:23.996587    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:24.065944 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:24.065905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:24.138615 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:24.138588    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:24.239108 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:24.239079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:34.718765 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:34.718687    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:35.069030 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:35.068929    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:35:56.101339 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:35:56.101244    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:02.464730 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:02.464699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:02.567345 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:02.567309    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:02.887202 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:02.887106    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:03.191693 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:03.191663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:03.392891 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:03.392862    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:06.166388 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.166353    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:06.273437 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.273403    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"]
Jul 22 21:36:06.273767 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.273752    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48838a6a-6627-445f-9873-26f8867f5a55" containerName="maas-api"
Jul 22 21:36:06.273767 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.273765    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="48838a6a-6627-445f-9873-26f8867f5a55" containerName="maas-api"
Jul 22 21:36:06.273920 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.273775    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72444975-d195-4e58-98f4-007979d764b9" containerName="revoke-keys"
Jul 22 21:36:06.273920 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.273781    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="72444975-d195-4e58-98f4-007979d764b9" containerName="revoke-keys"
Jul 22 21:36:06.273920 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.273838    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="72444975-d195-4e58-98f4-007979d764b9" containerName="revoke-keys"
Jul 22 21:36:06.273920 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.273850    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="48838a6a-6627-445f-9873-26f8867f5a55" containerName="maas-api"
Jul 22 21:36:06.276728 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.276708    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.279108 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.279078    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2f9a91-serving-cert\""
Jul 22 21:36:06.283152 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.283125    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:06.288102 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.288078    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"]
Jul 22 21:36:06.366106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.366069    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:06.437163 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.437093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.437163 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.437128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llbgv\" (UniqueName: \"kubernetes.io/projected/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-kube-api-access-llbgv\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.437340 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.437218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.538303 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.538278    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.538443 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.538344    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.538443 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.538378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llbgv\" (UniqueName: \"kubernetes.io/projected/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-kube-api-access-llbgv\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.538443 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:36:06.538419    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-serving-cert: secret "maas-api-e2e-inf-a-2f9a91-serving-cert" not found
Jul 22 21:36:06.538557 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:36:06.538475    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls podName:71dd9a3c-e195-42a1-b35e-a2ee50b5a819 nodeName:}" failed. No retries permitted until 2026-07-22 21:36:07.038460371 +0000 UTC m=+2349.439262334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls") pod "maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7" (UID: "71dd9a3c-e195-42a1-b35e-a2ee50b5a819") : secret "maas-api-e2e-inf-a-2f9a91-serving-cert" not found
Jul 22 21:36:06.539031 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.539014    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.546908 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.546875    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llbgv\" (UniqueName: \"kubernetes.io/projected/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-kube-api-access-llbgv\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:06.621020 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.620988    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:06.794589 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.794512    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:06.891514 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:06.891481    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:07.043072 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:07.043032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:07.045481 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:07.045416    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:07.188938 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:07.188908    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:07.324147 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:07.324075    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"]
Jul 22 21:36:07.327172 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:36:07.327147    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71dd9a3c_e195_42a1_b35e_a2ee50b5a819.slice/crio-92080408080ed9b63f48a4906545b98ce64ac27f4e77f431b38036496472ecb8 WatchSource:0}: Error finding container 92080408080ed9b63f48a4906545b98ce64ac27f4e77f431b38036496472ecb8: Status 404 returned error can't find the container with id 92080408080ed9b63f48a4906545b98ce64ac27f4e77f431b38036496472ecb8
Jul 22 21:36:08.257899 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:08.257859    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7" event={"ID":"71dd9a3c-e195-42a1-b35e-a2ee50b5a819","Type":"ContainerStarted","Data":"4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812"}
Jul 22 21:36:08.258296 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:08.257919    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7" event={"ID":"71dd9a3c-e195-42a1-b35e-a2ee50b5a819","Type":"ContainerStarted","Data":"92080408080ed9b63f48a4906545b98ce64ac27f4e77f431b38036496472ecb8"}
Jul 22 21:36:08.258296 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:08.257945    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:08.275364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:08.275317    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7" podStartSLOduration=1.761185428 podStartE2EDuration="2.27530356s" totalImagesPullingTime="514.118132ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:36:08.273697193 +0000 UTC m=+2350.674499180" watchObservedRunningTime="2026-07-22 21:36:08.27530356 +0000 UTC m=+2350.676105543"
Jul 22 21:36:10.710291 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:10.710257    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:11.074516 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:11.074437    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:11.387371 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:11.387293    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:11.644334 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:11.644303    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:14.169729 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.169684    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:14.266497 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.266467    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:14.268503 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.268478    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:36:14.404598 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.404568    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"]
Jul 22 21:36:14.408353 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.408329    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.410488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.410466    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-1dd516-serving-cert\""
Jul 22 21:36:14.420157 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.420095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"]
Jul 22 21:36:14.506919 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.506891    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs4g5\" (UniqueName: \"kubernetes.io/projected/68d493e8-614d-4793-9ffc-0818f740c61c-kube-api-access-hs4g5\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.507101 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.506950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d493e8-614d-4793-9ffc-0818f740c61c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.507101 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.507055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.514815 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.514785    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:14.608141 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.608114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d493e8-614d-4793-9ffc-0818f740c61c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.608293 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.608154    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.608293 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.608202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs4g5\" (UniqueName: \"kubernetes.io/projected/68d493e8-614d-4793-9ffc-0818f740c61c-kube-api-access-hs4g5\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.608429 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:36:14.608297    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-serving-cert: secret "maas-api-e2e-inf-b-1dd516-serving-cert" not found
Jul 22 21:36:14.608429 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:36:14.608366    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls podName:68d493e8-614d-4793-9ffc-0818f740c61c nodeName:}" failed. No retries permitted until 2026-07-22 21:36:15.108350061 +0000 UTC m=+2357.509152023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls") pod "maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" (UID: "68d493e8-614d-4793-9ffc-0818f740c61c") : secret "maas-api-e2e-inf-b-1dd516-serving-cert" not found
Jul 22 21:36:14.608809 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.608787    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d493e8-614d-4793-9ffc-0818f740c61c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.618598 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.618576    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs4g5\" (UniqueName: \"kubernetes.io/projected/68d493e8-614d-4793-9ffc-0818f740c61c-kube-api-access-hs4g5\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:14.760536 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.760453    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:14.921789 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:14.921757    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:15.024816 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:15.024736    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:15.112801 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:15.112763    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:15.115112 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:15.115085    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls\") pod \"maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:15.319583 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:15.319505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:15.452655 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:15.452598    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"]
Jul 22 21:36:15.455221 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:36:15.455188    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d493e8_614d_4793_9ffc_0818f740c61c.slice/crio-6c7203af168b61cb731055186a269006336cea1ee985a1e6a21ca0afc088a165 WatchSource:0}: Error finding container 6c7203af168b61cb731055186a269006336cea1ee985a1e6a21ca0afc088a165: Status 404 returned error can't find the container with id 6c7203af168b61cb731055186a269006336cea1ee985a1e6a21ca0afc088a165
Jul 22 21:36:16.296610 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:16.296573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" event={"ID":"68d493e8-614d-4793-9ffc-0818f740c61c","Type":"ContainerStarted","Data":"37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a"}
Jul 22 21:36:16.296780 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:16.296635    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" event={"ID":"68d493e8-614d-4793-9ffc-0818f740c61c","Type":"ContainerStarted","Data":"6c7203af168b61cb731055186a269006336cea1ee985a1e6a21ca0afc088a165"}
Jul 22 21:36:16.296780 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:16.296661    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:16.314191 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:16.314141    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" podStartSLOduration=1.9718028730000001 podStartE2EDuration="2.314126202s" totalImagesPullingTime="342.323329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:36:16.311784318 +0000 UTC m=+2358.712586302" watchObservedRunningTime="2026-07-22 21:36:16.314126202 +0000 UTC m=+2358.714928229"
Jul 22 21:36:18.752891 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:18.752853    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:21.530561 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:21.530523    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:22.309771 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:22.309743    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:36:27.634857 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:27.634824    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:27.735407 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:27.735377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:35.670313 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:35.670254    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:35.770510 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:35.770482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:49.905048 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.905014    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"]
Jul 22 21:36:49.909316 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.909299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:49.912928 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.912548    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1dd516\"/\"test-model-1dd516-kserve-self-signed-certs\""
Jul 22 21:36:49.912928 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.912642    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1dd516\"/\"kube-root-ca.crt\""
Jul 22 21:36:49.912928 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.912793    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1dd516\"/\"default-dockercfg-kl57c\""
Jul 22 21:36:49.913271 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.913009    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1dd516\"/\"openshift-service-ca.crt\""
Jul 22 21:36:49.916213 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.916171    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"]
Jul 22 21:36:49.934829 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.934808    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:49.973030 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.973001    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-home\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:49.973164 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.973043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-model-cache\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:49.973164 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.973076    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brjk5\" (UniqueName: \"kubernetes.io/projected/fe7b9242-0cf2-4a69-8759-cb90db720930-kube-api-access-brjk5\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:49.973164 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.973110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-tmp-dir\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:49.973368 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.973165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-dshm\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:49.973368 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:49.973232    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7b9242-0cf2-4a69-8759-cb90db720930-tls-certs\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074136 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074107    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-home\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074145    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-model-cache\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074190    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brjk5\" (UniqueName: \"kubernetes.io/projected/fe7b9242-0cf2-4a69-8759-cb90db720930-kube-api-access-brjk5\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-tmp-dir\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074281    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-dshm\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074322    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7b9242-0cf2-4a69-8759-cb90db720930-tls-certs\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074757 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074574    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-model-cache\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074950 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-tmp-dir\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.074950 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.074930    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-home\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.077524 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.077493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-dshm\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.077690 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.077668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7b9242-0cf2-4a69-8759-cb90db720930-tls-certs\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.081964 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.081945    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brjk5\" (UniqueName: \"kubernetes.io/projected/fe7b9242-0cf2-4a69-8759-cb90db720930-kube-api-access-brjk5\") pod \"test-model-1dd516-kserve-85f7c57995-pwq7w\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") " pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.220362 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.220304    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:36:50.353517 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.353478    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"]
Jul 22 21:36:50.356344 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:36:50.356231    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe7b9242_0cf2_4a69_8759_cb90db720930.slice/crio-71b0a90d2e50b64eae9e4595f70850a2cf880959c13b5cd1e95548e819bc8840 WatchSource:0}: Error finding container 71b0a90d2e50b64eae9e4595f70850a2cf880959c13b5cd1e95548e819bc8840: Status 404 returned error can't find the container with id 71b0a90d2e50b64eae9e4595f70850a2cf880959c13b5cd1e95548e819bc8840
Jul 22 21:36:50.462290 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.462250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w" event={"ID":"fe7b9242-0cf2-4a69-8759-cb90db720930","Type":"ContainerStarted","Data":"2696cf4b53787c612a977c07f2c739d6c37a1a56d2418bed944148cbb0d069e1"}
Jul 22 21:36:50.462438 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.462301    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w" event={"ID":"fe7b9242-0cf2-4a69-8759-cb90db720930","Type":"ContainerStarted","Data":"71b0a90d2e50b64eae9e4595f70850a2cf880959c13b5cd1e95548e819bc8840"}
Jul 22 21:36:50.480625 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:50.480542    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w" podStartSLOduration=1.480524964 podStartE2EDuration="1.480524964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:36:50.478342729 +0000 UTC m=+2392.879144710" watchObservedRunningTime="2026-07-22 21:36:50.480524964 +0000 UTC m=+2392.881326947"
Jul 22 21:36:58.135905 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:58.135855    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:36:58.152284 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:58.152256    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:36:58.219067 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:58.219032    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:36:58.287090 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:36:58.287060    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:00.220493 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:00.220466    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:37:00.220859 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:00.220525    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:37:00.232842 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:00.232815    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:37:00.524376 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:00.524297    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:37:28.437437 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:28.437406    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:29.499433 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:29.499397    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"]
Jul 22 21:37:29.509771 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:29.509744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:30.135164 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:30.135126    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:30.235775 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:30.235743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:40.087855 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.087820    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"]
Jul 22 21:37:40.088344 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.088050    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7" podUID="71dd9a3c-e195-42a1-b35e-a2ee50b5a819" containerName="maas-api" containerID="cri-o://4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812" gracePeriod=30
Jul 22 21:37:40.366327 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.366250    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:40.541077 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.541051    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:37:40.554673 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.554650    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls\") pod \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") "
Jul 22 21:37:40.554787 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.554702    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llbgv\" (UniqueName: \"kubernetes.io/projected/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-kube-api-access-llbgv\") pod \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") "
Jul 22 21:37:40.554787 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.554729    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-openshift-service-ca\") pod \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\" (UID: \"71dd9a3c-e195-42a1-b35e-a2ee50b5a819\") "
Jul 22 21:37:40.555160 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.555129    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71dd9a3c-e195-42a1-b35e-a2ee50b5a819" (UID: "71dd9a3c-e195-42a1-b35e-a2ee50b5a819"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:37:40.557255 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.557224    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-kube-api-access-llbgv" (OuterVolumeSpecName: "kube-api-access-llbgv") pod "71dd9a3c-e195-42a1-b35e-a2ee50b5a819" (UID: "71dd9a3c-e195-42a1-b35e-a2ee50b5a819"). InnerVolumeSpecName "kube-api-access-llbgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:37:40.558197 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.558136    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71dd9a3c-e195-42a1-b35e-a2ee50b5a819" (UID: "71dd9a3c-e195-42a1-b35e-a2ee50b5a819"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:37:40.655245 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.655218    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llbgv\" (UniqueName: \"kubernetes.io/projected/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-kube-api-access-llbgv\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:40.655346 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.655249    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:40.655346 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.655265    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71dd9a3c-e195-42a1-b35e-a2ee50b5a819-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:40.703121 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.703098    2559 generic.go:356] "Generic (PLEG): container finished" podID="71dd9a3c-e195-42a1-b35e-a2ee50b5a819" containerID="4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812" exitCode=0
Jul 22 21:37:40.703235 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.703140    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"
Jul 22 21:37:40.703235 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.703202    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7" event={"ID":"71dd9a3c-e195-42a1-b35e-a2ee50b5a819","Type":"ContainerDied","Data":"4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812"}
Jul 22 21:37:40.703316 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.703255    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7" event={"ID":"71dd9a3c-e195-42a1-b35e-a2ee50b5a819","Type":"ContainerDied","Data":"92080408080ed9b63f48a4906545b98ce64ac27f4e77f431b38036496472ecb8"}
Jul 22 21:37:40.703316 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.703276    2559 scope.go:117] "RemoveContainer" containerID="4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812"
Jul 22 21:37:40.712597 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.712582    2559 scope.go:117] "RemoveContainer" containerID="4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812"
Jul 22 21:37:40.712843 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:37:40.712824    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812\": container with ID starting with 4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812 not found: ID does not exist" containerID="4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812"
Jul 22 21:37:40.712904 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.712856    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812"} err="failed to get container status \"4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812\": rpc error: code = NotFound desc = could not find container \"4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812\": container with ID starting with 4f6cdf90c7b02417ab52523577f9906042982fcd0132258b24e730ce04920812 not found: ID does not exist"
Jul 22 21:37:40.722662 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.722642    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"]
Jul 22 21:37:40.725960 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:40.725941    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f9a91-5b768b4c55-995m7"]
Jul 22 21:37:42.076859 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:42.076828    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71dd9a3c-e195-42a1-b35e-a2ee50b5a819" path="/var/lib/kubelet/pods/71dd9a3c-e195-42a1-b35e-a2ee50b5a819/volumes"
Jul 22 21:37:44.519316 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.519273    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w" podUID="fe7b9242-0cf2-4a69-8759-cb90db720930" containerName="main" containerID="cri-o://2696cf4b53787c612a977c07f2c739d6c37a1a56d2418bed944148cbb0d069e1" gracePeriod=45
Jul 22 21:37:44.665002 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.664967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:44.731638 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.731600    2559 generic.go:356] "Generic (PLEG): container finished" podID="fe7b9242-0cf2-4a69-8759-cb90db720930" containerID="2696cf4b53787c612a977c07f2c739d6c37a1a56d2418bed944148cbb0d069e1" exitCode=0
Jul 22 21:37:44.731772 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.731675    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w" event={"ID":"fe7b9242-0cf2-4a69-8759-cb90db720930","Type":"ContainerDied","Data":"2696cf4b53787c612a977c07f2c739d6c37a1a56d2418bed944148cbb0d069e1"}
Jul 22 21:37:44.779970 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.779918    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:37:44.880007 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.879981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7b9242-0cf2-4a69-8759-cb90db720930-tls-certs\") pod \"fe7b9242-0cf2-4a69-8759-cb90db720930\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") "
Jul 22 21:37:44.880134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880021    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-model-cache\") pod \"fe7b9242-0cf2-4a69-8759-cb90db720930\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") "
Jul 22 21:37:44.880134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880042    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brjk5\" (UniqueName: \"kubernetes.io/projected/fe7b9242-0cf2-4a69-8759-cb90db720930-kube-api-access-brjk5\") pod \"fe7b9242-0cf2-4a69-8759-cb90db720930\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") "
Jul 22 21:37:44.880134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880069    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-dshm\") pod \"fe7b9242-0cf2-4a69-8759-cb90db720930\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") "
Jul 22 21:37:44.880306 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880203    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-tmp-dir\") pod \"fe7b9242-0cf2-4a69-8759-cb90db720930\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") "
Jul 22 21:37:44.880306 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880244    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-home\") pod \"fe7b9242-0cf2-4a69-8759-cb90db720930\" (UID: \"fe7b9242-0cf2-4a69-8759-cb90db720930\") "
Jul 22 21:37:44.880421 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880318    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe7b9242-0cf2-4a69-8759-cb90db720930" (UID: "fe7b9242-0cf2-4a69-8759-cb90db720930"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:37:44.880421 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880382    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe7b9242-0cf2-4a69-8759-cb90db720930" (UID: "fe7b9242-0cf2-4a69-8759-cb90db720930"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:37:44.880545 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880516    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-home" (OuterVolumeSpecName: "home") pod "fe7b9242-0cf2-4a69-8759-cb90db720930" (UID: "fe7b9242-0cf2-4a69-8759-cb90db720930"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:37:44.880596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880580    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-model-cache\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:44.880596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.880594    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-tmp-dir\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:44.882215 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.882193    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7b9242-0cf2-4a69-8759-cb90db720930-kube-api-access-brjk5" (OuterVolumeSpecName: "kube-api-access-brjk5") pod "fe7b9242-0cf2-4a69-8759-cb90db720930" (UID: "fe7b9242-0cf2-4a69-8759-cb90db720930"). InnerVolumeSpecName "kube-api-access-brjk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:37:44.882524 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.882506    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-dshm" (OuterVolumeSpecName: "dshm") pod "fe7b9242-0cf2-4a69-8759-cb90db720930" (UID: "fe7b9242-0cf2-4a69-8759-cb90db720930"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:37:44.882585 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.882523    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe7b9242-0cf2-4a69-8759-cb90db720930-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe7b9242-0cf2-4a69-8759-cb90db720930" (UID: "fe7b9242-0cf2-4a69-8759-cb90db720930"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:37:44.981756 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.981728    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-home\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:44.981756 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.981753    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe7b9242-0cf2-4a69-8759-cb90db720930-tls-certs\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:44.981900 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.981767    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brjk5\" (UniqueName: \"kubernetes.io/projected/fe7b9242-0cf2-4a69-8759-cb90db720930-kube-api-access-brjk5\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:44.981900 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:44.981781    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe7b9242-0cf2-4a69-8759-cb90db720930-dshm\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:37:45.073661 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:45.073586    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:45.738687 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:45.738661    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"
Jul 22 21:37:45.738687 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:45.738672    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w" event={"ID":"fe7b9242-0cf2-4a69-8759-cb90db720930","Type":"ContainerDied","Data":"71b0a90d2e50b64eae9e4595f70850a2cf880959c13b5cd1e95548e819bc8840"}
Jul 22 21:37:45.739243 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:45.738716    2559 scope.go:117] "RemoveContainer" containerID="2696cf4b53787c612a977c07f2c739d6c37a1a56d2418bed944148cbb0d069e1"
Jul 22 21:37:45.760028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:45.759998    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"]
Jul 22 21:37:45.764417 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:45.764396    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1dd516/test-model-1dd516-kserve-85f7c57995-pwq7w"]
Jul 22 21:37:46.076953 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:46.076881    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7b9242-0cf2-4a69-8759-cb90db720930" path="/var/lib/kubelet/pods/fe7b9242-0cf2-4a69-8759-cb90db720930/volumes"
Jul 22 21:37:51.031208 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:51.031162    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:51.135867 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:51.135841    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:51.310087 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:51.310013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:51.525907 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:51.525881    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:52.363028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:52.362991    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:37:52.501732 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:37:52.501700    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:02.067581 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.067541    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"]
Jul 22 21:38:02.069991 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.068842    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" podUID="68d493e8-614d-4793-9ffc-0818f740c61c" containerName="maas-api" containerID="cri-o://37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a" gracePeriod=30
Jul 22 21:38:02.101638 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.101602    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:02.325809 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.325750    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:38:02.411519 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.411492    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs4g5\" (UniqueName: \"kubernetes.io/projected/68d493e8-614d-4793-9ffc-0818f740c61c-kube-api-access-hs4g5\") pod \"68d493e8-614d-4793-9ffc-0818f740c61c\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") "
Jul 22 21:38:02.411654 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.411527    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls\") pod \"68d493e8-614d-4793-9ffc-0818f740c61c\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") "
Jul 22 21:38:02.411654 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.411560    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d493e8-614d-4793-9ffc-0818f740c61c-openshift-service-ca\") pod \"68d493e8-614d-4793-9ffc-0818f740c61c\" (UID: \"68d493e8-614d-4793-9ffc-0818f740c61c\") "
Jul 22 21:38:02.411914 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.411889    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d493e8-614d-4793-9ffc-0818f740c61c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68d493e8-614d-4793-9ffc-0818f740c61c" (UID: "68d493e8-614d-4793-9ffc-0818f740c61c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:38:02.413562 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.413539    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68d493e8-614d-4793-9ffc-0818f740c61c" (UID: "68d493e8-614d-4793-9ffc-0818f740c61c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:38:02.413672 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.413628    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d493e8-614d-4793-9ffc-0818f740c61c-kube-api-access-hs4g5" (OuterVolumeSpecName: "kube-api-access-hs4g5") pod "68d493e8-614d-4793-9ffc-0818f740c61c" (UID: "68d493e8-614d-4793-9ffc-0818f740c61c"). InnerVolumeSpecName "kube-api-access-hs4g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:38:02.512364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.512336    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs4g5\" (UniqueName: \"kubernetes.io/projected/68d493e8-614d-4793-9ffc-0818f740c61c-kube-api-access-hs4g5\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:38:02.512364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.512357    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68d493e8-614d-4793-9ffc-0818f740c61c-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:38:02.512503 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.512369    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68d493e8-614d-4793-9ffc-0818f740c61c-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:38:02.821951 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.821915    2559 generic.go:356] "Generic (PLEG): container finished" podID="68d493e8-614d-4793-9ffc-0818f740c61c" containerID="37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a" exitCode=0
Jul 22 21:38:02.822103 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.821959    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"
Jul 22 21:38:02.822103 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.821999    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" event={"ID":"68d493e8-614d-4793-9ffc-0818f740c61c","Type":"ContainerDied","Data":"37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a"}
Jul 22 21:38:02.822103 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.822052    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" event={"ID":"68d493e8-614d-4793-9ffc-0818f740c61c","Type":"ContainerDied","Data":"6c7203af168b61cb731055186a269006336cea1ee985a1e6a21ca0afc088a165"}
Jul 22 21:38:02.822103 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.822074    2559 scope.go:117] "RemoveContainer" containerID="37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a"
Jul 22 21:38:02.831060 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.831044    2559 scope.go:117] "RemoveContainer" containerID="37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a"
Jul 22 21:38:02.831343 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:02.831321    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a\": container with ID starting with 37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a not found: ID does not exist" containerID="37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a"
Jul 22 21:38:02.831404 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.831351    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a"} err="failed to get container status \"37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a\": rpc error: code = NotFound desc = could not find container \"37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a\": container with ID starting with 37da1defae9467635e5bcc7fd95c8f11c5292d25910f4015e7426c2d046dab0a not found: ID does not exist"
Jul 22 21:38:02.842769 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.842747    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"]
Jul 22 21:38:02.846526 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:02.846504    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g"]
Jul 22 21:38:04.077369 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:04.077331    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68d493e8-614d-4793-9ffc-0818f740c61c" path="/var/lib/kubelet/pods/68d493e8-614d-4793-9ffc-0818f740c61c/volumes"
Jul 22 21:38:05.304455 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:05.304413    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1dd516-798bff8b9c-6h84g" podUID="68d493e8-614d-4793-9ffc-0818f740c61c" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.48:8443/health\": context deadline exceeded"
Jul 22 21:38:06.696457 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:06.696426    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:07.072577 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:07.072501    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:12.846836 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:12.846804    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:12.918609 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:12.918580    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:13.146917 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:13.146888    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:13.402390 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:13.402313    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:13.801887 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:13.801811    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:14.060069 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:14.059995    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:14.174690 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:14.174660    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:16.765226 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.765171    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:16.845403 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.845363    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"]
Jul 22 21:38:16.846144 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846120    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71dd9a3c-e195-42a1-b35e-a2ee50b5a819" containerName="maas-api"
Jul 22 21:38:16.846337 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846311    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="71dd9a3c-e195-42a1-b35e-a2ee50b5a819" containerName="maas-api"
Jul 22 21:38:16.846461 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846355    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe7b9242-0cf2-4a69-8759-cb90db720930" containerName="main"
Jul 22 21:38:16.846461 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846364    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe7b9242-0cf2-4a69-8759-cb90db720930" containerName="main"
Jul 22 21:38:16.846461 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846377    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68d493e8-614d-4793-9ffc-0818f740c61c" containerName="maas-api"
Jul 22 21:38:16.846461 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846386    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="68d493e8-614d-4793-9ffc-0818f740c61c" containerName="maas-api"
Jul 22 21:38:16.846611 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846481    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="68d493e8-614d-4793-9ffc-0818f740c61c" containerName="maas-api"
Jul 22 21:38:16.846611 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846493    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe7b9242-0cf2-4a69-8759-cb90db720930" containerName="main"
Jul 22 21:38:16.846611 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.846503    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="71dd9a3c-e195-42a1-b35e-a2ee50b5a819" containerName="maas-api"
Jul 22 21:38:16.850058 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.850030    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:16.852935 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.852913    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5d7596-serving-cert\""
Jul 22 21:38:16.862321 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.862301    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"]
Jul 22 21:38:16.872513 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.872487    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:16.918281 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.918253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:16.918390 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.918311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c7159-72a2-4ec8-8870-00266b249d74-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:16.918435 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.918403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw9fg\" (UniqueName: \"kubernetes.io/projected/771c7159-72a2-4ec8-8870-00266b249d74-kube-api-access-xw9fg\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:16.967373 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:16.967348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:17.019127 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.019067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9fg\" (UniqueName: \"kubernetes.io/projected/771c7159-72a2-4ec8-8870-00266b249d74-kube-api-access-xw9fg\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.019127 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.019100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.019299 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.019155    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c7159-72a2-4ec8-8870-00266b249d74-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.019299 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:17.019270    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-serving-cert: secret "maas-api-e2e-api-a-5d7596-serving-cert" not found
Jul 22 21:38:17.019376 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:17.019344    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls podName:771c7159-72a2-4ec8-8870-00266b249d74 nodeName:}" failed. No retries permitted until 2026-07-22 21:38:17.519316136 +0000 UTC m=+2479.920118105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls") pod "maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g" (UID: "771c7159-72a2-4ec8-8870-00266b249d74") : secret "maas-api-e2e-api-a-5d7596-serving-cert" not found
Jul 22 21:38:17.019830 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.019812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c7159-72a2-4ec8-8870-00266b249d74-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.030000 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.029979    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw9fg\" (UniqueName: \"kubernetes.io/projected/771c7159-72a2-4ec8-8870-00266b249d74-kube-api-access-xw9fg\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.261871 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.261838    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:17.512114 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.512079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:17.523944 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.523915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.526387 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.526363    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls\") pod \"maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.612039 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.612005    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:17.761753 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.761722    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:17.896921 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:17.896898    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"]
Jul 22 21:38:17.899223 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:38:17.899186    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771c7159_72a2_4ec8_8870_00266b249d74.slice/crio-2db47bc1ee05a4db31f860adaa44d6a474b7350738c9bcd294a8145b56934a11 WatchSource:0}: Error finding container 2db47bc1ee05a4db31f860adaa44d6a474b7350738c9bcd294a8145b56934a11: Status 404 returned error can't find the container with id 2db47bc1ee05a4db31f860adaa44d6a474b7350738c9bcd294a8145b56934a11
Jul 22 21:38:18.904451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:18.904413    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g" event={"ID":"771c7159-72a2-4ec8-8870-00266b249d74","Type":"ContainerStarted","Data":"6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6"}
Jul 22 21:38:18.904807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:18.904600    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:18.904807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:18.904646    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g" event={"ID":"771c7159-72a2-4ec8-8870-00266b249d74","Type":"ContainerStarted","Data":"2db47bc1ee05a4db31f860adaa44d6a474b7350738c9bcd294a8145b56934a11"}
Jul 22 21:38:18.920249 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:18.920210    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g" podStartSLOduration=2.26127071 podStartE2EDuration="2.920197511s" totalImagesPullingTime="658.926801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:38:18.918944265 +0000 UTC m=+2481.319746260" watchObservedRunningTime="2026-07-22 21:38:18.920197511 +0000 UTC m=+2481.320999488"
Jul 22 21:38:21.234043 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:21.233997    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:21.683658 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:21.683631    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:22.116820 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:22.116740    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:22.321398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:22.321355    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:24.824930 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.824887    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:24.905773 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.905744    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"]
Jul 22 21:38:24.909519 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.909493    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:24.911127 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.911085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:24.911908 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.911888    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-af2d1d-serving-cert\""
Jul 22 21:38:24.914983 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.914964    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:24.920293 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.920270    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"]
Jul 22 21:38:24.984891 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.984862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-openshift-service-ca\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:24.985031 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.984896    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c94xq\" (UniqueName: \"kubernetes.io/projected/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-kube-api-access-c94xq\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:24.985031 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.984985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:24.991238 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:24.991217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:25.085772 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.085704    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:25.085915 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.085796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-openshift-service-ca\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:25.085915 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.085824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c94xq\" (UniqueName: \"kubernetes.io/projected/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-kube-api-access-c94xq\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:25.085915 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:25.085858    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-serving-cert: secret "maas-api-e2e-api-b-af2d1d-serving-cert" not found
Jul 22 21:38:25.086082 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:25.085942    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls podName:1792c6dc-166d-46bd-9f7e-cea5015e4f0a nodeName:}" failed. No retries permitted until 2026-07-22 21:38:25.585922223 +0000 UTC m=+2487.986724188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls") pod "maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" (UID: "1792c6dc-166d-46bd-9f7e-cea5015e4f0a") : secret "maas-api-e2e-api-b-af2d1d-serving-cert" not found
Jul 22 21:38:25.086414 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.086397    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-openshift-service-ca\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:25.097023 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.096949    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c94xq\" (UniqueName: \"kubernetes.io/projected/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-kube-api-access-c94xq\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:25.554472 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.554439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:25.590601 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.590572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:25.590804 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:25.590753    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-serving-cert: secret "maas-api-e2e-api-b-af2d1d-serving-cert" not found
Jul 22 21:38:25.590885 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:25.590823    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls podName:1792c6dc-166d-46bd-9f7e-cea5015e4f0a nodeName:}" failed. No retries permitted until 2026-07-22 21:38:26.59080406 +0000 UTC m=+2488.991606021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls") pod "maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" (UID: "1792c6dc-166d-46bd-9f7e-cea5015e4f0a") : secret "maas-api-e2e-api-b-af2d1d-serving-cert" not found
Jul 22 21:38:25.657292 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.657266    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:25.759461 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:25.759433    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:26.599445 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:26.599404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:26.601686 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:26.601663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls\") pod \"maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:26.722269 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:26.722239    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:26.855632 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:26.855605    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"]
Jul 22 21:38:26.857903 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:38:26.857864    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1792c6dc_166d_46bd_9f7e_cea5015e4f0a.slice/crio-f9e4301ab3665fcef0b0e66aa7a259bdb1ba8d5042118f2e71db45ecb8d36544 WatchSource:0}: Error finding container f9e4301ab3665fcef0b0e66aa7a259bdb1ba8d5042118f2e71db45ecb8d36544: Status 404 returned error can't find the container with id f9e4301ab3665fcef0b0e66aa7a259bdb1ba8d5042118f2e71db45ecb8d36544
Jul 22 21:38:26.859693 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:26.859671    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 21:38:26.948581 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:26.948549    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" event={"ID":"1792c6dc-166d-46bd-9f7e-cea5015e4f0a","Type":"ContainerStarted","Data":"f9e4301ab3665fcef0b0e66aa7a259bdb1ba8d5042118f2e71db45ecb8d36544"}
Jul 22 21:38:27.956925 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:27.956892    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" event={"ID":"1792c6dc-166d-46bd-9f7e-cea5015e4f0a","Type":"ContainerStarted","Data":"4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563"}
Jul 22 21:38:27.957302 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:27.956953    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:27.975724 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:27.975673    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" podStartSLOduration=3.595444771 podStartE2EDuration="3.975659639s" totalImagesPullingTime="380.214868ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:38:27.973497555 +0000 UTC m=+2490.374299538" watchObservedRunningTime="2026-07-22 21:38:27.975659639 +0000 UTC m=+2490.376461621"
Jul 22 21:38:29.327895 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:29.327860    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:33.967053 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:33.967017    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:38:37.823997 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:37.823967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:37.924397 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:37.924357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:47.233135 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:47.233085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:47.341108 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:47.341079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:52.472911 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.472825    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"]
Jul 22 21:38:52.473314 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.473098    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g" podUID="771c7159-72a2-4ec8-8870-00266b249d74" containerName="maas-api" containerID="cri-o://6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6" gracePeriod=30
Jul 22 21:38:52.491858 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.491837    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:52.721881 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.721855    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:52.906004 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.905978    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c7159-72a2-4ec8-8870-00266b249d74-openshift-service-ca\") pod \"771c7159-72a2-4ec8-8870-00266b249d74\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") "
Jul 22 21:38:52.906159 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.906075    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw9fg\" (UniqueName: \"kubernetes.io/projected/771c7159-72a2-4ec8-8870-00266b249d74-kube-api-access-xw9fg\") pod \"771c7159-72a2-4ec8-8870-00266b249d74\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") "
Jul 22 21:38:52.906159 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.906103    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls\") pod \"771c7159-72a2-4ec8-8870-00266b249d74\" (UID: \"771c7159-72a2-4ec8-8870-00266b249d74\") "
Jul 22 21:38:52.906406 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.906381    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/771c7159-72a2-4ec8-8870-00266b249d74-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "771c7159-72a2-4ec8-8870-00266b249d74" (UID: "771c7159-72a2-4ec8-8870-00266b249d74"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:38:52.908208 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.908165    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "771c7159-72a2-4ec8-8870-00266b249d74" (UID: "771c7159-72a2-4ec8-8870-00266b249d74"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:38:52.908208 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:52.908196    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/771c7159-72a2-4ec8-8870-00266b249d74-kube-api-access-xw9fg" (OuterVolumeSpecName: "kube-api-access-xw9fg") pod "771c7159-72a2-4ec8-8870-00266b249d74" (UID: "771c7159-72a2-4ec8-8870-00266b249d74"). InnerVolumeSpecName "kube-api-access-xw9fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:38:53.007420 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.007396    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/771c7159-72a2-4ec8-8870-00266b249d74-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:38:53.007420 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.007419    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw9fg\" (UniqueName: \"kubernetes.io/projected/771c7159-72a2-4ec8-8870-00266b249d74-kube-api-access-xw9fg\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:38:53.007569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.007428    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/771c7159-72a2-4ec8-8870-00266b249d74-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:38:53.075714 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.075688    2559 generic.go:356] "Generic (PLEG): container finished" podID="771c7159-72a2-4ec8-8870-00266b249d74" containerID="6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6" exitCode=0
Jul 22 21:38:53.075840 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.075735    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"
Jul 22 21:38:53.075840 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.075742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g" event={"ID":"771c7159-72a2-4ec8-8870-00266b249d74","Type":"ContainerDied","Data":"6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6"}
Jul 22 21:38:53.075840 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.075777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g" event={"ID":"771c7159-72a2-4ec8-8870-00266b249d74","Type":"ContainerDied","Data":"2db47bc1ee05a4db31f860adaa44d6a474b7350738c9bcd294a8145b56934a11"}
Jul 22 21:38:53.075840 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.075803    2559 scope.go:117] "RemoveContainer" containerID="6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6"
Jul 22 21:38:53.084496 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.084480    2559 scope.go:117] "RemoveContainer" containerID="6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6"
Jul 22 21:38:53.084714 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:38:53.084695    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6\": container with ID starting with 6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6 not found: ID does not exist" containerID="6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6"
Jul 22 21:38:53.084762 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.084721    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6"} err="failed to get container status \"6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6\": rpc error: code = NotFound desc = could not find container \"6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6\": container with ID starting with 6bbe3ec243d781c6e46cf0278d101bef2ab481e81cef52b22e1c5ebd19e5f5b6 not found: ID does not exist"
Jul 22 21:38:53.096384 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.096363    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"]
Jul 22 21:38:53.102277 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:53.102259    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d7596-74599fbc76-gwh9g"]
Jul 22 21:38:54.077300 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:54.077265    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="771c7159-72a2-4ec8-8870-00266b249d74" path="/var/lib/kubelet/pods/771c7159-72a2-4ec8-8870-00266b249d74/volumes"
Jul 22 21:38:57.411720 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:57.411687    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:38:57.776006 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:38:57.775926    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:03.641204 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:03.641135    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:03.734283 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:03.734252    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:03.905991 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:03.905902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:04.116457 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:04.116417    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:19.652755 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:19.652718    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"]
Jul 22 21:39:19.653653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:19.653617    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" podUID="1792c6dc-166d-46bd-9f7e-cea5015e4f0a" containerName="maas-api" containerID="cri-o://4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563" gracePeriod=30
Jul 22 21:39:19.687969 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:19.687945    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:19.900733 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:19.900711    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:39:20.014428 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.014358    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls\") pod \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") "
Jul 22 21:39:20.014428 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.014416    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c94xq\" (UniqueName: \"kubernetes.io/projected/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-kube-api-access-c94xq\") pod \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") "
Jul 22 21:39:20.014591 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.014452    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-openshift-service-ca\") pod \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\" (UID: \"1792c6dc-166d-46bd-9f7e-cea5015e4f0a\") "
Jul 22 21:39:20.014836 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.014806    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1792c6dc-166d-46bd-9f7e-cea5015e4f0a" (UID: "1792c6dc-166d-46bd-9f7e-cea5015e4f0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:39:20.016375 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.016350    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1792c6dc-166d-46bd-9f7e-cea5015e4f0a" (UID: "1792c6dc-166d-46bd-9f7e-cea5015e4f0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:39:20.016454 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.016368    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-kube-api-access-c94xq" (OuterVolumeSpecName: "kube-api-access-c94xq") pod "1792c6dc-166d-46bd-9f7e-cea5015e4f0a" (UID: "1792c6dc-166d-46bd-9f7e-cea5015e4f0a"). InnerVolumeSpecName "kube-api-access-c94xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:39:20.115555 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.115526    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:39:20.115648 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.115556    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:39:20.115648 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.115570    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c94xq\" (UniqueName: \"kubernetes.io/projected/1792c6dc-166d-46bd-9f7e-cea5015e4f0a-kube-api-access-c94xq\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:39:20.195270 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.195241    2559 generic.go:356] "Generic (PLEG): container finished" podID="1792c6dc-166d-46bd-9f7e-cea5015e4f0a" containerID="4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563" exitCode=0
Jul 22 21:39:20.195373 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.195282    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"
Jul 22 21:39:20.195373 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.195320    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" event={"ID":"1792c6dc-166d-46bd-9f7e-cea5015e4f0a","Type":"ContainerDied","Data":"4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563"}
Jul 22 21:39:20.195451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.195375    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm" event={"ID":"1792c6dc-166d-46bd-9f7e-cea5015e4f0a","Type":"ContainerDied","Data":"f9e4301ab3665fcef0b0e66aa7a259bdb1ba8d5042118f2e71db45ecb8d36544"}
Jul 22 21:39:20.195451 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.195397    2559 scope.go:117] "RemoveContainer" containerID="4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563"
Jul 22 21:39:20.203859 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.203842    2559 scope.go:117] "RemoveContainer" containerID="4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563"
Jul 22 21:39:20.204111 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:39:20.204091    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563\": container with ID starting with 4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563 not found: ID does not exist" containerID="4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563"
Jul 22 21:39:20.204163 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.204118    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563"} err="failed to get container status \"4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563\": rpc error: code = NotFound desc = could not find container \"4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563\": container with ID starting with 4dad648f4e84970f3b4c3fd3207d2bcdc5483dec730009d6731304ae66679563 not found: ID does not exist"
Jul 22 21:39:20.210477 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.210455    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"]
Jul 22 21:39:20.214077 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:20.214057    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-af2d1d-89b948dd9-6vbwm"]
Jul 22 21:39:22.076912 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:22.076884    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1792c6dc-166d-46bd-9f7e-cea5015e4f0a" path="/var/lib/kubelet/pods/1792c6dc-166d-46bd-9f7e-cea5015e4f0a/volumes"
Jul 22 21:39:24.316970 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:24.316934    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:24.719169 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:24.719134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:30.421563 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:30.421532    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:30.522266 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:30.522235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:30.695363 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:30.695266    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:30.917110 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:30.917076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:31.267048 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:31.267019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:31.546869 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:31.546776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:31.649981 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:31.649946    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:34.145797 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.145750    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:34.203547 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.203510    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"]
Jul 22 21:39:34.203966 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.203949    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="771c7159-72a2-4ec8-8870-00266b249d74" containerName="maas-api"
Jul 22 21:39:34.203966 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.203967    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="771c7159-72a2-4ec8-8870-00266b249d74" containerName="maas-api"
Jul 22 21:39:34.204088 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.203974    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1792c6dc-166d-46bd-9f7e-cea5015e4f0a" containerName="maas-api"
Jul 22 21:39:34.204088 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.203979    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1792c6dc-166d-46bd-9f7e-cea5015e4f0a" containerName="maas-api"
Jul 22 21:39:34.204088 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.204046    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1792c6dc-166d-46bd-9f7e-cea5015e4f0a" containerName="maas-api"
Jul 22 21:39:34.204088 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.204060    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="771c7159-72a2-4ec8-8870-00266b249d74" containerName="maas-api"
Jul 22 21:39:34.208495 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.208473    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.210926 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.210906    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-644efc-serving-cert\""
Jul 22 21:39:34.216640 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.216619    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"]
Jul 22 21:39:34.233633 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.233608    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:34.307549 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.307524    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:34.324674 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.324650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxzp\" (UniqueName: \"kubernetes.io/projected/1ae73507-603b-48af-92ff-ee75a02811ae-kube-api-access-rbxzp\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.324808 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.324686    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae73507-603b-48af-92ff-ee75a02811ae-maas-api-tls\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.324808 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.324772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae73507-603b-48af-92ff-ee75a02811ae-openshift-service-ca\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.426134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.426055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae73507-603b-48af-92ff-ee75a02811ae-openshift-service-ca\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.426134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.426113    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxzp\" (UniqueName: \"kubernetes.io/projected/1ae73507-603b-48af-92ff-ee75a02811ae-kube-api-access-rbxzp\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.426370 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.426148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae73507-603b-48af-92ff-ee75a02811ae-maas-api-tls\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.426669 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.426649    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae73507-603b-48af-92ff-ee75a02811ae-openshift-service-ca\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.428525 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.428505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae73507-603b-48af-92ff-ee75a02811ae-maas-api-tls\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.434538 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.434516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxzp\" (UniqueName: \"kubernetes.io/projected/1ae73507-603b-48af-92ff-ee75a02811ae-kube-api-access-rbxzp\") pod \"maas-api-e2e-shared-a-644efc-58c6758697-8lbbv\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.441063 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.441036    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:34.520067 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.520044    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:34.676037 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.676003    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"]
Jul 22 21:39:34.679815 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:39:34.679786    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae73507_603b_48af_92ff_ee75a02811ae.slice/crio-d62d84a4309f873bdd2e6de3eb219c5f8b56858a32bda98ac5ba01ebfb82b464 WatchSource:0}: Error finding container d62d84a4309f873bdd2e6de3eb219c5f8b56858a32bda98ac5ba01ebfb82b464: Status 404 returned error can't find the container with id d62d84a4309f873bdd2e6de3eb219c5f8b56858a32bda98ac5ba01ebfb82b464
Jul 22 21:39:34.773149 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.773116    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:34.873124 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:34.873099    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:35.265529 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:35.265496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv" event={"ID":"1ae73507-603b-48af-92ff-ee75a02811ae","Type":"ContainerStarted","Data":"8212273571f395e380dac8f4df9576022a01ecf9e0c3089f21baffb63cafbd7b"}
Jul 22 21:39:35.265847 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:35.265556    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv" event={"ID":"1ae73507-603b-48af-92ff-ee75a02811ae","Type":"ContainerStarted","Data":"d62d84a4309f873bdd2e6de3eb219c5f8b56858a32bda98ac5ba01ebfb82b464"}
Jul 22 21:39:35.265847 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:35.265595    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:35.281653 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:35.281609    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv" podStartSLOduration=0.835608776 podStartE2EDuration="1.281593431s" totalImagesPullingTime="445.984655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:39:35.280001385 +0000 UTC m=+2557.680803371" watchObservedRunningTime="2026-07-22 21:39:35.281593431 +0000 UTC m=+2557.682395413"
Jul 22 21:39:38.657145 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:38.657106    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:38.974138 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:38.974062    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:39.280299 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:39.280203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:39.379658 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:39.379618    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:41.275618 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:41.275592    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:39:42.259664 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.259630    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:42.344349 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.344316    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"]
Jul 22 21:39:42.345802 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.345771    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:42.348329 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.348307    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.351094 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.351073    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bfd0ef-serving-cert\""
Jul 22 21:39:42.358677 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.358656    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"]
Jul 22 21:39:42.408140 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.408116    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:42.490786 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.490757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-maas-api-tls\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.490922 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.490811    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxmbc\" (UniqueName: \"kubernetes.io/projected/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-kube-api-access-lxmbc\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.490922 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.490838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.592267 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.592172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-maas-api-tls\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.592267 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.592259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxmbc\" (UniqueName: \"kubernetes.io/projected/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-kube-api-access-lxmbc\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.592423 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.592286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.592917 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.592891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.594755 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.594735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-maas-api-tls\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.600294 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.600277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxmbc\" (UniqueName: \"kubernetes.io/projected/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-kube-api-access-lxmbc\") pod \"maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.642118 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.642071    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:42.660097 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.660080    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:42.821883 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.821860    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"]
Jul 22 21:39:42.824271 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:39:42.824237    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ad39a6_0526_464d_ab4e_9a4ddb3b2d14.slice/crio-5d734269838200dea5a75ce86faa4bf1b34f0d93a74c5ddbddaa873a0bcae07b WatchSource:0}: Error finding container 5d734269838200dea5a75ce86faa4bf1b34f0d93a74c5ddbddaa873a0bcae07b: Status 404 returned error can't find the container with id 5d734269838200dea5a75ce86faa4bf1b34f0d93a74c5ddbddaa873a0bcae07b
Jul 22 21:39:42.997337 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:42.997308    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:43.097710 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:43.097683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:43.303201 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:43.303116    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2" event={"ID":"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14","Type":"ContainerStarted","Data":"5d734269838200dea5a75ce86faa4bf1b34f0d93a74c5ddbddaa873a0bcae07b"}
Jul 22 21:39:44.314753 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:44.314717    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2" event={"ID":"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14","Type":"ContainerStarted","Data":"ae2752ea6cb5ba3c1af94ac6c62bb7fe025241f0bb6fd409d95a4cb1378ea22b"}
Jul 22 21:39:44.315134 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:44.314817    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:44.331620 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:44.331582    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2" podStartSLOduration=1.359678456 podStartE2EDuration="2.331570083s" totalImagesPullingTime="971.891627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:39:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:39:44.329764137 +0000 UTC m=+2566.730566123" watchObservedRunningTime="2026-07-22 21:39:44.331570083 +0000 UTC m=+2566.732372087"
Jul 22 21:39:46.743888 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:46.743853    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:50.325391 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:50.325356    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:39:55.583394 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:55.583358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:55.694600 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:55.694567    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:39:58.246171 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:39:58.246140    2559 scope.go:117] "RemoveContainer" containerID="732249a51f0a87bf7badb4f1fbeff6f969a9e372bd62b7459972e90d836fa349"
Jul 22 21:40:00.328730 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:00.328702    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:40:04.655194 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:04.655144    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:40:04.755297 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:04.755247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:40:13.085346 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:13.085307    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:40:13.186548 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:13.186518    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:40:14.905146 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.905116    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"]
Jul 22 21:40:14.908617 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.908597    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:14.911523 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.911501    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"auth-test-model-bfd0ef-kserve-self-signed-certs\""
Jul 22 21:40:14.911655 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.911522    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"openshift-service-ca.crt\""
Jul 22 21:40:14.911655 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.911542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"default-dockercfg-48ntt\""
Jul 22 21:40:14.911655 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.911629    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"kube-root-ca.crt\""
Jul 22 21:40:14.917599 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.917574    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"]
Jul 22 21:40:14.954560 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.954534    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-home\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:14.954678 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.954569    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tmp-dir\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:14.954678 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.954633    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tls-certs\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:14.954803 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.954699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-model-cache\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:14.954803 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.954744    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-dshm\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:14.954803 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:14.954788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf6c6\" (UniqueName: \"kubernetes.io/projected/0be1c1bb-8a40-48bf-b5ec-077b30be366c-kube-api-access-cf6c6\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.001062 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.001039    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:40:15.055223 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-home\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055354 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tmp-dir\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055354 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055295    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tls-certs\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055354 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-model-cache\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-dshm\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf6c6\" (UniqueName: \"kubernetes.io/projected/0be1c1bb-8a40-48bf-b5ec-077b30be366c-kube-api-access-cf6c6\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055553    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-home\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055744 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055622    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tmp-dir\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.055812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.055745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-model-cache\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.057543 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.057517    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-dshm\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.057900 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.057881    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tls-certs\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.064576 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.064557    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf6c6\" (UniqueName: \"kubernetes.io/projected/0be1c1bb-8a40-48bf-b5ec-077b30be366c-kube-api-access-cf6c6\") pod \"auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") " pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.218503 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.218432    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:15.351118 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.351092    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"]
Jul 22 21:40:15.353911 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:40:15.353884    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0be1c1bb_8a40_48bf_b5ec_077b30be366c.slice/crio-d1053cebccb90406a6af27ba5f6252e6e77e2bb4d8d244f30a9e482e214d0b5f WatchSource:0}: Error finding container d1053cebccb90406a6af27ba5f6252e6e77e2bb4d8d244f30a9e482e214d0b5f: Status 404 returned error can't find the container with id d1053cebccb90406a6af27ba5f6252e6e77e2bb4d8d244f30a9e482e214d0b5f
Jul 22 21:40:15.467995 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.467963    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c" event={"ID":"0be1c1bb-8a40-48bf-b5ec-077b30be366c","Type":"ContainerStarted","Data":"ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9"}
Jul 22 21:40:15.468123 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.468009    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c" event={"ID":"0be1c1bb-8a40-48bf-b5ec-077b30be366c","Type":"ContainerStarted","Data":"d1053cebccb90406a6af27ba5f6252e6e77e2bb4d8d244f30a9e482e214d0b5f"}
Jul 22 21:40:15.484817 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:15.484738    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c" podStartSLOduration=1.4847230630000001 podStartE2EDuration="1.484723063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:40:15.483139174 +0000 UTC m=+2597.883941157" watchObservedRunningTime="2026-07-22 21:40:15.484723063 +0000 UTC m=+2597.885525073"
Jul 22 21:40:25.218875 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:25.218793    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:25.218875 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:25.218835    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:25.231098 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:25.231074    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:40:25.524734 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:40:25.524662    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:41:12.647038 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:41:12.647003    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:41:30.709494 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:41:30.709459    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:41:44.927120 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:41:44.927084    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:41:58.172449 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:41:58.172369    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:41:58.180112 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:41:58.180090    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:42:14.503040 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:14.503007    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:42:15.435623 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:15.435584    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"]
Jul 22 21:42:15.456061 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:15.456028    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:42:30.459064 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.458994    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c" podUID="0be1c1bb-8a40-48bf-b5ec-077b30be366c" containerName="main" containerID="cri-o://ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9" gracePeriod=45
Jul 22 21:42:30.715146 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.715092    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:42:30.809459 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809427    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-model-cache\") pod \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") "
Jul 22 21:42:30.809604 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809496    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-dshm\") pod \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") "
Jul 22 21:42:30.809604 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809545    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-home\") pod \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") "
Jul 22 21:42:30.809604 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809586    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tmp-dir\") pod \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") "
Jul 22 21:42:30.809784 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809612    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tls-certs\") pod \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") "
Jul 22 21:42:30.809784 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809637    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf6c6\" (UniqueName: \"kubernetes.io/projected/0be1c1bb-8a40-48bf-b5ec-077b30be366c-kube-api-access-cf6c6\") pod \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\" (UID: \"0be1c1bb-8a40-48bf-b5ec-077b30be366c\") "
Jul 22 21:42:30.809784 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809678    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-model-cache" (OuterVolumeSpecName: "model-cache") pod "0be1c1bb-8a40-48bf-b5ec-077b30be366c" (UID: "0be1c1bb-8a40-48bf-b5ec-077b30be366c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:42:30.809784 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809771    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-home" (OuterVolumeSpecName: "home") pod "0be1c1bb-8a40-48bf-b5ec-077b30be366c" (UID: "0be1c1bb-8a40-48bf-b5ec-077b30be366c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:42:30.809964 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809878    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0be1c1bb-8a40-48bf-b5ec-077b30be366c" (UID: "0be1c1bb-8a40-48bf-b5ec-077b30be366c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:42:30.809964 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809951    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-model-cache\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:42:30.810048 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.809973    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-home\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:42:30.811704 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.811683    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-dshm" (OuterVolumeSpecName: "dshm") pod "0be1c1bb-8a40-48bf-b5ec-077b30be366c" (UID: "0be1c1bb-8a40-48bf-b5ec-077b30be366c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:42:30.811771 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.811755    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be1c1bb-8a40-48bf-b5ec-077b30be366c-kube-api-access-cf6c6" (OuterVolumeSpecName: "kube-api-access-cf6c6") pod "0be1c1bb-8a40-48bf-b5ec-077b30be366c" (UID: "0be1c1bb-8a40-48bf-b5ec-077b30be366c"). InnerVolumeSpecName "kube-api-access-cf6c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:42:30.811824 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.811762    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0be1c1bb-8a40-48bf-b5ec-077b30be366c" (UID: "0be1c1bb-8a40-48bf-b5ec-077b30be366c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:42:30.911210 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.911168    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tls-certs\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:42:30.911310 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.911220    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf6c6\" (UniqueName: \"kubernetes.io/projected/0be1c1bb-8a40-48bf-b5ec-077b30be366c-kube-api-access-cf6c6\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:42:30.911310 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.911231    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-dshm\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:42:30.911310 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:30.911239    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0be1c1bb-8a40-48bf-b5ec-077b30be366c-tmp-dir\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:42:31.101971 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.101900    2559 generic.go:356] "Generic (PLEG): container finished" podID="0be1c1bb-8a40-48bf-b5ec-077b30be366c" containerID="ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9" exitCode=0
Jul 22 21:42:31.101971 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.101941    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"
Jul 22 21:42:31.102124 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.101984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c" event={"ID":"0be1c1bb-8a40-48bf-b5ec-077b30be366c","Type":"ContainerDied","Data":"ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9"}
Jul 22 21:42:31.102124 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.102022    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c" event={"ID":"0be1c1bb-8a40-48bf-b5ec-077b30be366c","Type":"ContainerDied","Data":"d1053cebccb90406a6af27ba5f6252e6e77e2bb4d8d244f30a9e482e214d0b5f"}
Jul 22 21:42:31.102124 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.102040    2559 scope.go:117] "RemoveContainer" containerID="ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9"
Jul 22 21:42:31.111433 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.111413    2559 scope.go:117] "RemoveContainer" containerID="ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9"
Jul 22 21:42:31.111666 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:42:31.111649    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9\": container with ID starting with ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9 not found: ID does not exist" containerID="ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9"
Jul 22 21:42:31.111721 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.111673    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9"} err="failed to get container status \"ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9\": rpc error: code = NotFound desc = could not find container \"ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9\": container with ID starting with ca11fed4df571d93f8540612959498b8e5e2114928468924b67801ffe8d8ffb9 not found: ID does not exist"
Jul 22 21:42:31.122657 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.122636    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"]
Jul 22 21:42:31.128002 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:31.127984    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/auth-test-model-bfd0ef-kserve-7cf86f485f-7h84c"]
Jul 22 21:42:32.077234 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:32.077201    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be1c1bb-8a40-48bf-b5ec-077b30be366c" path="/var/lib/kubelet/pods/0be1c1bb-8a40-48bf-b5ec-077b30be366c/volumes"
Jul 22 21:42:32.441433 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:32.441396    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:42:52.597965 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:52.597937    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:42:52.654615 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:42:52.654596    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:43:01.800704 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.800667    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"]
Jul 22 21:43:01.801079 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.801038    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be1c1bb-8a40-48bf-b5ec-077b30be366c" containerName="main"
Jul 22 21:43:01.801079 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.801050    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be1c1bb-8a40-48bf-b5ec-077b30be366c" containerName="main"
Jul 22 21:43:01.801163 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.801117    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be1c1bb-8a40-48bf-b5ec-077b30be366c" containerName="main"
Jul 22 21:43:01.805596 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.805571    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.808693 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.808665    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"kube-root-ca.crt\""
Jul 22 21:43:01.808826 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.808666    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"sub-test-model-bfd0ef-kserve-self-signed-certs\""
Jul 22 21:43:01.808826 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.808709    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"openshift-service-ca.crt\""
Jul 22 21:43:01.808826 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.808746    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bfd0ef\"/\"default-dockercfg-48ntt\""
Jul 22 21:43:01.813510 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.813488    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"]
Jul 22 21:43:01.838610 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.838588    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-home\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.838735 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.838623    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-dshm\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.838735 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.838639    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tls-certs\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.838735 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.838656    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tmp-dir\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.838735 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.838713    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk4db\" (UniqueName: \"kubernetes.io/projected/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-kube-api-access-sk4db\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.838907 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.838748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-model-cache\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.840375 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.840342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:43:01.940089 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-home\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940252 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-dshm\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940252 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tls-certs\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940252 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940134    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tmp-dir\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940252 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940170    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk4db\" (UniqueName: \"kubernetes.io/projected/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-kube-api-access-sk4db\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940252 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940227    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-model-cache\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940568 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940534    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tmp-dir\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940734 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-model-cache\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.940734 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.940711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-home\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.942674 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.942647    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-dshm\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.943338 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.943317    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tls-certs\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:01.951227 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:01.951206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk4db\" (UniqueName: \"kubernetes.io/projected/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-kube-api-access-sk4db\") pod \"sub-test-model-bfd0ef-kserve-64f8698fc-5wncg\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") " pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:02.116454 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:02.116382    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:02.252705 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:02.252680    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"]
Jul 22 21:43:02.254751 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:43:02.254720    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef45bbe7_b692_45b8_8b4a_9e7229436dc6.slice/crio-4e1c14deef6a2015008c4f0db761508d516f7d832131607fa04ff9c0ac74e2bd WatchSource:0}: Error finding container 4e1c14deef6a2015008c4f0db761508d516f7d832131607fa04ff9c0ac74e2bd: Status 404 returned error can't find the container with id 4e1c14deef6a2015008c4f0db761508d516f7d832131607fa04ff9c0ac74e2bd
Jul 22 21:43:03.250816 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:03.250784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg" event={"ID":"ef45bbe7-b692-45b8-8b4a-9e7229436dc6","Type":"ContainerStarted","Data":"ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57"}
Jul 22 21:43:03.251333 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:03.250830    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg" event={"ID":"ef45bbe7-b692-45b8-8b4a-9e7229436dc6","Type":"ContainerStarted","Data":"4e1c14deef6a2015008c4f0db761508d516f7d832131607fa04ff9c0ac74e2bd"}
Jul 22 21:43:03.268560 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:03.268510    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg" podStartSLOduration=2.268495427 podStartE2EDuration="2.268495427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:43:03.266254174 +0000 UTC m=+2765.667056164" watchObservedRunningTime="2026-07-22 21:43:03.268495427 +0000 UTC m=+2765.669297410"
Jul 22 21:43:12.117076 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:12.117044    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:12.117571 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:12.117093    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:12.129237 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:12.129215    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:12.301360 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:12.301335    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:43:53.430002 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:53.429970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:43:54.231569 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:54.231536    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"]
Jul 22 21:43:54.249166 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:43:54.249115    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:44:08.034969 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:08.034936    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:44:09.255437 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.255395    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg" podUID="ef45bbe7-b692-45b8-8b4a-9e7229436dc6" containerName="main" containerID="cri-o://ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57" gracePeriod=45
Jul 22 21:44:09.501930 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.501907    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:44:09.564168 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.564099    2559 generic.go:356] "Generic (PLEG): container finished" podID="ef45bbe7-b692-45b8-8b4a-9e7229436dc6" containerID="ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57" exitCode=0
Jul 22 21:44:09.564168 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.564143    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"
Jul 22 21:44:09.564168 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.564156    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg" event={"ID":"ef45bbe7-b692-45b8-8b4a-9e7229436dc6","Type":"ContainerDied","Data":"ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57"}
Jul 22 21:44:09.564433 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.564208    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg" event={"ID":"ef45bbe7-b692-45b8-8b4a-9e7229436dc6","Type":"ContainerDied","Data":"4e1c14deef6a2015008c4f0db761508d516f7d832131607fa04ff9c0ac74e2bd"}
Jul 22 21:44:09.564433 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.564226    2559 scope.go:117] "RemoveContainer" containerID="ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57"
Jul 22 21:44:09.571239 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571220    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-dshm\") pod \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") "
Jul 22 21:44:09.571351 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571250    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk4db\" (UniqueName: \"kubernetes.io/projected/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-kube-api-access-sk4db\") pod \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") "
Jul 22 21:44:09.571351 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571293    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tmp-dir\") pod \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") "
Jul 22 21:44:09.571351 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571316    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-model-cache\") pod \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") "
Jul 22 21:44:09.571521 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571366    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-home\") pod \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") "
Jul 22 21:44:09.571521 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571393    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tls-certs\") pod \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\" (UID: \"ef45bbe7-b692-45b8-8b4a-9e7229436dc6\") "
Jul 22 21:44:09.571647 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571617    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-home" (OuterVolumeSpecName: "home") pod "ef45bbe7-b692-45b8-8b4a-9e7229436dc6" (UID: "ef45bbe7-b692-45b8-8b4a-9e7229436dc6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:44:09.571707 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571641    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef45bbe7-b692-45b8-8b4a-9e7229436dc6" (UID: "ef45bbe7-b692-45b8-8b4a-9e7229436dc6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:44:09.571707 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.571630    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef45bbe7-b692-45b8-8b4a-9e7229436dc6" (UID: "ef45bbe7-b692-45b8-8b4a-9e7229436dc6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:44:09.572660 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.572637    2559 scope.go:117] "RemoveContainer" containerID="ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57"
Jul 22 21:44:09.572945 ip-10-0-131-38 kubenswrapper[2559]: E0722 21:44:09.572923    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57\": container with ID starting with ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57 not found: ID does not exist" containerID="ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57"
Jul 22 21:44:09.573029 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.572951    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57"} err="failed to get container status \"ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57\": rpc error: code = NotFound desc = could not find container \"ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57\": container with ID starting with ac1b3a0e3719f9388359c9b30d6f2a5ec06134b9aedc616669443fa58df4ab57 not found: ID does not exist"
Jul 22 21:44:09.573310 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.573289    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-dshm" (OuterVolumeSpecName: "dshm") pod "ef45bbe7-b692-45b8-8b4a-9e7229436dc6" (UID: "ef45bbe7-b692-45b8-8b4a-9e7229436dc6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 21:44:09.573482 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.573461    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef45bbe7-b692-45b8-8b4a-9e7229436dc6" (UID: "ef45bbe7-b692-45b8-8b4a-9e7229436dc6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:44:09.573529 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.573498    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-kube-api-access-sk4db" (OuterVolumeSpecName: "kube-api-access-sk4db") pod "ef45bbe7-b692-45b8-8b4a-9e7229436dc6" (UID: "ef45bbe7-b692-45b8-8b4a-9e7229436dc6"). InnerVolumeSpecName "kube-api-access-sk4db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:44:09.672289 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.672256    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-home\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:44:09.672289 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.672287    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tls-certs\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:44:09.672289 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.672297    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-dshm\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:44:09.672454 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.672305    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk4db\" (UniqueName: \"kubernetes.io/projected/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-kube-api-access-sk4db\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:44:09.672454 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.672312    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-tmp-dir\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:44:09.672454 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.672320    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef45bbe7-b692-45b8-8b4a-9e7229436dc6-model-cache\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:44:09.884910 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.884881    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"]
Jul 22 21:44:09.888838 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:09.888815    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bfd0ef/sub-test-model-bfd0ef-kserve-64f8698fc-5wncg"]
Jul 22 21:44:10.077927 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:10.077895    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef45bbe7-b692-45b8-8b4a-9e7229436dc6" path="/var/lib/kubelet/pods/ef45bbe7-b692-45b8-8b4a-9e7229436dc6/volumes"
Jul 22 21:44:27.054364 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:27.054332    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:44:28.170722 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:28.170693    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:44:41.181351 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:41.181312    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:44:56.435618 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:56.435545    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:44:56.535770 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:44:56.535743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:45:00.139616 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.139583    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"]
Jul 22 21:45:00.140158 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.140141    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef45bbe7-b692-45b8-8b4a-9e7229436dc6" containerName="main"
Jul 22 21:45:00.140224 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.140162    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef45bbe7-b692-45b8-8b4a-9e7229436dc6" containerName="main"
Jul 22 21:45:00.140293 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.140276    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef45bbe7-b692-45b8-8b4a-9e7229436dc6" containerName="main"
Jul 22 21:45:00.149794 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.149709    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"]
Jul 22 21:45:00.149928 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.149821    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"
Jul 22 21:45:00.154767 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.154746    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz5vc\""
Jul 22 21:45:00.156229 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.156206    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg"]
Jul 22 21:45:00.156336 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.156279    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"
Jul 22 21:45:00.160008 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.159986    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"]
Jul 22 21:45:00.160120 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.160026    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"]
Jul 22 21:45:00.160120 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.160048    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg"]
Jul 22 21:45:00.160228 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.160128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg"
Jul 22 21:45:00.165698 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.165679    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"
Jul 22 21:45:00.174084 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.174064    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"
Jul 22 21:45:00.205770 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.205744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg"
Jul 22 21:45:00.335195 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.335116    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"]
Jul 22 21:45:00.384668 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.384647    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg"]
Jul 22 21:45:00.386347 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:45:00.386324    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9e18cd_798c_4309_9b70_b0e07739a6fd.slice/crio-6400ce43f056e7fb2a85ff44d55b81472fdbda0fcc388d7083d17e60c79b6e7a WatchSource:0}: Error finding container 6400ce43f056e7fb2a85ff44d55b81472fdbda0fcc388d7083d17e60c79b6e7a: Status 404 returned error can't find the container with id 6400ce43f056e7fb2a85ff44d55b81472fdbda0fcc388d7083d17e60c79b6e7a
Jul 22 21:45:00.566102 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.566065    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"]
Jul 22 21:45:00.568033 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:45:00.568008    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff0f9a7_e5a2_414e_a128_b88990bbc3f5.slice/crio-15427085c22e8db2c089f46cca8fbf3dc696356b757b5bf08e1da00c8db9f12d WatchSource:0}: Error finding container 15427085c22e8db2c089f46cca8fbf3dc696356b757b5bf08e1da00c8db9f12d: Status 404 returned error can't find the container with id 15427085c22e8db2c089f46cca8fbf3dc696356b757b5bf08e1da00c8db9f12d
Jul 22 21:45:00.802031 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.801999    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8" event={"ID":"3ff0f9a7-e5a2-414e-a128-b88990bbc3f5","Type":"ContainerStarted","Data":"15427085c22e8db2c089f46cca8fbf3dc696356b757b5bf08e1da00c8db9f12d"}
Jul 22 21:45:00.803479 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.803448    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg" event={"ID":"2b9e18cd-798c-4309-9b70-b0e07739a6fd","Type":"ContainerStarted","Data":"ec4904f92b4c57a9492c438925d6284551ed8b828e67f85f6658be1641b5ee8f"}
Jul 22 21:45:00.803618 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.803502    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg" event={"ID":"2b9e18cd-798c-4309-9b70-b0e07739a6fd","Type":"ContainerStarted","Data":"6400ce43f056e7fb2a85ff44d55b81472fdbda0fcc388d7083d17e60c79b6e7a"}
Jul 22 21:45:00.804683 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.804660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl" event={"ID":"5b2e870b-b608-4740-bf96-f6b027018885","Type":"ContainerStarted","Data":"aeb13cf7f79dbf25a77e2e68f452adb4f2f0a105ffb3df0099c476ba9b2e13ca"}
Jul 22 21:45:00.804796 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.804705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl" event={"ID":"5b2e870b-b608-4740-bf96-f6b027018885","Type":"ContainerStarted","Data":"acc9ba9a752b0cd6a545caaadd97ad88c7959190762f6265b48ef24f3a275a12"}
Jul 22 21:45:00.816899 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.816852    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg" podStartSLOduration=0.81684051 podStartE2EDuration="816.84051ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:45:00.815908097 +0000 UTC m=+2883.216710102" watchObservedRunningTime="2026-07-22 21:45:00.81684051 +0000 UTC m=+2883.217642842"
Jul 22 21:45:00.832457 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:00.832413    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl" podStartSLOduration=0.8324007 podStartE2EDuration="832.4007ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:45:00.830835855 +0000 UTC m=+2883.231637839" watchObservedRunningTime="2026-07-22 21:45:00.8324007 +0000 UTC m=+2883.233202682"
Jul 22 21:45:01.811621 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:01.811587    2559 generic.go:356] "Generic (PLEG): container finished" podID="2b9e18cd-798c-4309-9b70-b0e07739a6fd" containerID="ec4904f92b4c57a9492c438925d6284551ed8b828e67f85f6658be1641b5ee8f" exitCode=0
Jul 22 21:45:01.812026 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:01.811668    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg" event={"ID":"2b9e18cd-798c-4309-9b70-b0e07739a6fd","Type":"ContainerDied","Data":"ec4904f92b4c57a9492c438925d6284551ed8b828e67f85f6658be1641b5ee8f"}
Jul 22 21:45:01.813075 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:01.813052    2559 generic.go:356] "Generic (PLEG): container finished" podID="5b2e870b-b608-4740-bf96-f6b027018885" containerID="aeb13cf7f79dbf25a77e2e68f452adb4f2f0a105ffb3df0099c476ba9b2e13ca" exitCode=0
Jul 22 21:45:01.813200 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:01.813080    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl" event={"ID":"5b2e870b-b608-4740-bf96-f6b027018885","Type":"ContainerDied","Data":"aeb13cf7f79dbf25a77e2e68f452adb4f2f0a105ffb3df0099c476ba9b2e13ca"}
Jul 22 21:45:01.814415 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:01.814397    2559 generic.go:356] "Generic (PLEG): container finished" podID="3ff0f9a7-e5a2-414e-a128-b88990bbc3f5" containerID="3e5df3ab138e446b86b49a557b848349d06a5a15fe1539254e354000c423c2b0" exitCode=0
Jul 22 21:45:01.814485 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:01.814423    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8" event={"ID":"3ff0f9a7-e5a2-414e-a128-b88990bbc3f5","Type":"ContainerDied","Data":"3e5df3ab138e446b86b49a557b848349d06a5a15fe1539254e354000c423c2b0"}
Jul 22 21:45:02.319276 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:02.319241    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:45:02.971160 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:02.971122    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg"
Jul 22 21:45:03.016850 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.016832    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"
Jul 22 21:45:03.027727 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.027710    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"
Jul 22 21:45:03.828788 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.828652    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"
Jul 22 21:45:03.828788 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.828669    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8" event={"ID":"3ff0f9a7-e5a2-414e-a128-b88990bbc3f5","Type":"ContainerDied","Data":"15427085c22e8db2c089f46cca8fbf3dc696356b757b5bf08e1da00c8db9f12d"}
Jul 22 21:45:03.829052 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.828865    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15427085c22e8db2c089f46cca8fbf3dc696356b757b5bf08e1da00c8db9f12d"
Jul 22 21:45:03.830321 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.830301    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg"
Jul 22 21:45:03.830459 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.830343    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745945-7zpzg" event={"ID":"2b9e18cd-798c-4309-9b70-b0e07739a6fd","Type":"ContainerDied","Data":"6400ce43f056e7fb2a85ff44d55b81472fdbda0fcc388d7083d17e60c79b6e7a"}
Jul 22 21:45:03.830459 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.830382    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6400ce43f056e7fb2a85ff44d55b81472fdbda0fcc388d7083d17e60c79b6e7a"
Jul 22 21:45:03.831807 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.831792    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"
Jul 22 21:45:03.831939 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.831784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl" event={"ID":"5b2e870b-b608-4740-bf96-f6b027018885","Type":"ContainerDied","Data":"acc9ba9a752b0cd6a545caaadd97ad88c7959190762f6265b48ef24f3a275a12"}
Jul 22 21:45:03.831939 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:03.831916    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acc9ba9a752b0cd6a545caaadd97ad88c7959190762f6265b48ef24f3a275a12"
Jul 22 21:45:05.385456 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:05.385419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:45:05.493053 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:05.493020    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:45:13.927261 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:13.927230    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:45:14.027846 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:14.027810    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:45:51.631860 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:51.631829    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:45:52.408438 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:45:52.408407    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:12.060447 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:12.060414    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:12.155778 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:12.155752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:21.852150 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:21.852061    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"]
Jul 22 21:46:21.852749 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:21.852330    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv" podUID="1ae73507-603b-48af-92ff-ee75a02811ae" containerName="maas-api" containerID="cri-o://8212273571f395e380dac8f4df9576022a01ecf9e0c3089f21baffb63cafbd7b" gracePeriod=30
Jul 22 21:46:21.882293 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:21.882260    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:21.882944 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:21.882920    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"]
Jul 22 21:46:21.884885 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:21.884863    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-644efc-29745945-t88g8"]
Jul 22 21:46:22.077290 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.077260    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ff0f9a7-e5a2-414e-a128-b88990bbc3f5" path="/var/lib/kubelet/pods/3ff0f9a7-e5a2-414e-a128-b88990bbc3f5/volumes"
Jul 22 21:46:22.200385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.200348    2559 generic.go:356] "Generic (PLEG): container finished" podID="1ae73507-603b-48af-92ff-ee75a02811ae" containerID="8212273571f395e380dac8f4df9576022a01ecf9e0c3089f21baffb63cafbd7b" exitCode=0
Jul 22 21:46:22.200547 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.200414    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv" event={"ID":"1ae73507-603b-48af-92ff-ee75a02811ae","Type":"ContainerDied","Data":"8212273571f395e380dac8f4df9576022a01ecf9e0c3089f21baffb63cafbd7b"}
Jul 22 21:46:22.302515 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.302495    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:46:22.464155 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.464122    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbxzp\" (UniqueName: \"kubernetes.io/projected/1ae73507-603b-48af-92ff-ee75a02811ae-kube-api-access-rbxzp\") pod \"1ae73507-603b-48af-92ff-ee75a02811ae\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") "
Jul 22 21:46:22.464341 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.464198    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae73507-603b-48af-92ff-ee75a02811ae-openshift-service-ca\") pod \"1ae73507-603b-48af-92ff-ee75a02811ae\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") "
Jul 22 21:46:22.464341 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.464240    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae73507-603b-48af-92ff-ee75a02811ae-maas-api-tls\") pod \"1ae73507-603b-48af-92ff-ee75a02811ae\" (UID: \"1ae73507-603b-48af-92ff-ee75a02811ae\") "
Jul 22 21:46:22.464535 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.464511    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae73507-603b-48af-92ff-ee75a02811ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ae73507-603b-48af-92ff-ee75a02811ae" (UID: "1ae73507-603b-48af-92ff-ee75a02811ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:46:22.466142 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.466121    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae73507-603b-48af-92ff-ee75a02811ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ae73507-603b-48af-92ff-ee75a02811ae" (UID: "1ae73507-603b-48af-92ff-ee75a02811ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:46:22.466238 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.466219    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ae73507-603b-48af-92ff-ee75a02811ae-kube-api-access-rbxzp" (OuterVolumeSpecName: "kube-api-access-rbxzp") pod "1ae73507-603b-48af-92ff-ee75a02811ae" (UID: "1ae73507-603b-48af-92ff-ee75a02811ae"). InnerVolumeSpecName "kube-api-access-rbxzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:46:22.564868 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.564842    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ae73507-603b-48af-92ff-ee75a02811ae-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:46:22.564868 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.564865    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbxzp\" (UniqueName: \"kubernetes.io/projected/1ae73507-603b-48af-92ff-ee75a02811ae-kube-api-access-rbxzp\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:46:22.565002 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:22.564879    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae73507-603b-48af-92ff-ee75a02811ae-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:46:23.206527 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:23.206495    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"
Jul 22 21:46:23.206977 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:23.206537    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv" event={"ID":"1ae73507-603b-48af-92ff-ee75a02811ae","Type":"ContainerDied","Data":"d62d84a4309f873bdd2e6de3eb219c5f8b56858a32bda98ac5ba01ebfb82b464"}
Jul 22 21:46:23.206977 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:23.206597    2559 scope.go:117] "RemoveContainer" containerID="8212273571f395e380dac8f4df9576022a01ecf9e0c3089f21baffb63cafbd7b"
Jul 22 21:46:23.227105 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:23.227081    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"]
Jul 22 21:46:23.231024 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:23.231003    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-644efc-58c6758697-8lbbv"]
Jul 22 21:46:24.077166 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:24.077132    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ae73507-603b-48af-92ff-ee75a02811ae" path="/var/lib/kubelet/pods/1ae73507-603b-48af-92ff-ee75a02811ae/volumes"
Jul 22 21:46:26.635396 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:26.635362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:26.927925 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:26.927891    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:32.849960 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:32.849921    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:32.917673 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:32.917639    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:33.121610 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:33.121521    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:33.385599 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:33.385518    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:34.229567 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:34.229533    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:34.338046 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:34.338011    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:43.915979 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:43.915946    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"]
Jul 22 21:46:43.916632 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:43.916210    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2" podUID="79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" containerName="maas-api" containerID="cri-o://ae2752ea6cb5ba3c1af94ac6c62bb7fe025241f0bb6fd409d95a4cb1378ea22b" gracePeriod=30
Jul 22 21:46:43.943883 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:43.943852    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:43.948269 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:43.948240    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"]
Jul 22 21:46:43.951114 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:43.951081    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bfd0ef-29745945-275xl"]
Jul 22 21:46:44.077202 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.077151    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2e870b-b608-4740-bf96-f6b027018885" path="/var/lib/kubelet/pods/5b2e870b-b608-4740-bf96-f6b027018885/volumes"
Jul 22 21:46:44.313954 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.313924    2559 generic.go:356] "Generic (PLEG): container finished" podID="79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" containerID="ae2752ea6cb5ba3c1af94ac6c62bb7fe025241f0bb6fd409d95a4cb1378ea22b" exitCode=0
Jul 22 21:46:44.314102 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.313969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2" event={"ID":"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14","Type":"ContainerDied","Data":"ae2752ea6cb5ba3c1af94ac6c62bb7fe025241f0bb6fd409d95a4cb1378ea22b"}
Jul 22 21:46:44.365488 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.365466    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:46:44.537367 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.537295    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxmbc\" (UniqueName: \"kubernetes.io/projected/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-kube-api-access-lxmbc\") pod \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") "
Jul 22 21:46:44.537367 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.537336    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-maas-api-tls\") pod \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") "
Jul 22 21:46:44.537536 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.537485    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-openshift-service-ca\") pod \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\" (UID: \"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14\") "
Jul 22 21:46:44.537811 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.537784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" (UID: "79ad39a6-0526-464d-ab4e-9a4ddb3b2d14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 21:46:44.539398 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.539375    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" (UID: "79ad39a6-0526-464d-ab4e-9a4ddb3b2d14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 21:46:44.539511 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.539426    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-kube-api-access-lxmbc" (OuterVolumeSpecName: "kube-api-access-lxmbc") pod "79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" (UID: "79ad39a6-0526-464d-ab4e-9a4ddb3b2d14"). InnerVolumeSpecName "kube-api-access-lxmbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 21:46:44.638818 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.638786    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxmbc\" (UniqueName: \"kubernetes.io/projected/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-kube-api-access-lxmbc\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:46:44.638818 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.638819    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-maas-api-tls\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:46:44.638818 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:44.638829    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14-openshift-service-ca\") on node \"ip-10-0-131-38.ec2.internal\" DevicePath \"\""
Jul 22 21:46:45.319789 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:45.319761    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"
Jul 22 21:46:45.320266 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:45.319787    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2" event={"ID":"79ad39a6-0526-464d-ab4e-9a4ddb3b2d14","Type":"ContainerDied","Data":"5d734269838200dea5a75ce86faa4bf1b34f0d93a74c5ddbddaa873a0bcae07b"}
Jul 22 21:46:45.320266 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:45.319850    2559 scope.go:117] "RemoveContainer" containerID="ae2752ea6cb5ba3c1af94ac6c62bb7fe025241f0bb6fd409d95a4cb1378ea22b"
Jul 22 21:46:45.341002 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:45.340980    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"]
Jul 22 21:46:45.344769 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:45.344748    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bfd0ef-69f8d499f5-k8fd2"]
Jul 22 21:46:46.076494 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:46.076462    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" path="/var/lib/kubelet/pods/79ad39a6-0526-464d-ab4e-9a4ddb3b2d14/volumes"
Jul 22 21:46:48.590410 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:48.590370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:49.002697 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:49.002668    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:54.871385 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:54.871352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:54.925103 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:54.925070    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:55.128524 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:55.128437    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:55.365829 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:55.365792    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:46:58.207142 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:58.207114    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:46:58.213531 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:46:58.213512    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9tbv4_9cfd05ea-f2a7-4e43-8d07-d83d48c5c945/ovn-acl-logging/0.log"
Jul 22 21:47:01.479992 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:01.479961    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ss7nx_6bf69893-ad00-45fa-9c76-a6b5dfffc588/manager/0.log"
Jul 22 21:47:01.598190 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:01.598154    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d5898b76-64r57_f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc/manager/0.log"
Jul 22 21:47:01.708254 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:01.708231    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lbs79_26b03ead-8ee3-4bbc-bc12-0124700f9a9d/server/0.log"
Jul 22 21:47:01.825900 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:01.825842    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fzq8x_786ef4eb-1d55-4cdc-af35-895719287f5e/manager/0.log"
Jul 22 21:47:02.066119 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:02.066098    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846644596-wddxk_c03d398d-e157-45a0-892d-cca57f13c587/copy-manifests/0.log"
Jul 22 21:47:02.073950 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:02.073917    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846644596-wddxk_c03d398d-e157-45a0-892d-cca57f13c587/manager/0.log"
Jul 22 21:47:02.603704 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:02.603669    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-859cb6f49-tzkpt_0f384b6b-3afb-40a8-b86b-47bfd1578dad/maas-api/0.log"
Jul 22 21:47:02.826911 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:02.826889    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745930-4phlg_5ec19b5d-0abf-4281-94da-b5369091aa43/cleanup/0.log"
Jul 22 21:47:02.934537 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:02.934513    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745945-7zpzg_2b9e18cd-798c-4309-9b70-b0e07739a6fd/cleanup/0.log"
Jul 22 21:47:04.440152 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:04.440116    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h57jl_6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c/manager/0.log"
Jul 22 21:47:04.911279 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:04.911250    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bdpjs_5cdcd6b2-d522-483f-b478-357cf6ed1e44/manager/0.log"
Jul 22 21:47:05.615908 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:05.615882    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pxxm_f6d7a11e-a9a1-4103-97d4-685fc52f8434/discovery/0.log"
Jul 22 21:47:05.839460 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:05.839435    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6766dd8-th24k_b18e2476-a2d3-4c30-bf0b-464aafab41f6/kube-auth-proxy/0.log"
Jul 22 21:47:06.073339 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:06.073316    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-r4486_401d4b9c-7c49-4975-a310-aa2c958a7a5f/payload-pre-processing/0.log"
Jul 22 21:47:06.191106 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:06.191085    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hbp5n_0cbce6c2-b781-4112-b55e-ca19806a301f/payload-processing/0.log"
Jul 22 21:47:06.763904 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:06.763879    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-57spt_deab4530-6f24-409c-9bf0-804f2e0d9591/main/0.log"
Jul 22 21:47:07.111769 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:07.111698    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-qcbhh_629f7b9a-cb35-444d-94cd-33de243a0dcc/main/0.log"
Jul 22 21:47:12.710046 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:12.710013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:47:12.786842 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:12.786792    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log" limit=1024
Jul 22 21:47:15.958446 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:15.958419    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8wfk7_8c7417f6-3df6-418d-b90c-5e7f49f92fe9/global-pull-secret-syncer/0.log"
Jul 22 21:47:16.051162 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:16.051130    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9rkmp_3512e037-e77e-48b8-9b2a-46365e7fa80d/konnectivity-agent/0.log"
Jul 22 21:47:16.144832 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:16.144800    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-38.ec2.internal_ae44a980b4949927e8289bffa3154c50/haproxy/0.log"
Jul 22 21:47:22.101249 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:22.101216    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h57jl_6b5a3b6c-2dbe-40c4-8b85-f2ed16b10f1c/manager/0.log"
Jul 22 21:47:22.393117 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:22.393072    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bdpjs_5cdcd6b2-d522-483f-b478-357cf6ed1e44/manager/0.log"
Jul 22 21:47:24.547761 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:24.547725    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wwc8_a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777/node-exporter/0.log"
Jul 22 21:47:24.569531 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:24.569510    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wwc8_a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777/kube-rbac-proxy/0.log"
Jul 22 21:47:24.590917 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:24.590897    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wwc8_a29e2c2a-53a6-4fe4-8f1d-b92dbaca2777/init-textfile/0.log"
Jul 22 21:47:26.366726 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.366694    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"]
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367059    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2e870b-b608-4740-bf96-f6b027018885" containerName="cleanup"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367072    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2e870b-b608-4740-bf96-f6b027018885" containerName="cleanup"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367079    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b9e18cd-798c-4309-9b70-b0e07739a6fd" containerName="cleanup"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367085    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b9e18cd-798c-4309-9b70-b0e07739a6fd" containerName="cleanup"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367102    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" containerName="maas-api"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367107    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" containerName="maas-api"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367113    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff0f9a7-e5a2-414e-a128-b88990bbc3f5" containerName="cleanup"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367118    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff0f9a7-e5a2-414e-a128-b88990bbc3f5" containerName="cleanup"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367127    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ae73507-603b-48af-92ff-ee75a02811ae" containerName="maas-api"
Jul 22 21:47:26.367130 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367132    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ae73507-603b-48af-92ff-ee75a02811ae" containerName="maas-api"
Jul 22 21:47:26.367512 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367198    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff0f9a7-e5a2-414e-a128-b88990bbc3f5" containerName="cleanup"
Jul 22 21:47:26.367512 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367206    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ad39a6-0526-464d-ab4e-9a4ddb3b2d14" containerName="maas-api"
Jul 22 21:47:26.367512 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367213    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ae73507-603b-48af-92ff-ee75a02811ae" containerName="maas-api"
Jul 22 21:47:26.367512 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367220    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2e870b-b608-4740-bf96-f6b027018885" containerName="cleanup"
Jul 22 21:47:26.367512 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.367225    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b9e18cd-798c-4309-9b70-b0e07739a6fd" containerName="cleanup"
Jul 22 21:47:26.370256 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.370241    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.372512 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.372490    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rv7sd\"/\"kube-root-ca.crt\""
Jul 22 21:47:26.372601 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.372511    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rv7sd\"/\"default-dockercfg-cpdmh\""
Jul 22 21:47:26.373089 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.373065    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rv7sd\"/\"openshift-service-ca.crt\""
Jul 22 21:47:26.378739 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.378713    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"]
Jul 22 21:47:26.453697 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.453672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-podres\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.453812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.453702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzb45\" (UniqueName: \"kubernetes.io/projected/f7933497-5e6f-44d3-9983-17aa99d2a22c-kube-api-access-lzb45\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.453812 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.453729    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-lib-modules\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.453930 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.453817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-proc\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.453930 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.453864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-sys\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554426 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-sys\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554539 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554459    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-podres\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554539 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzb45\" (UniqueName: \"kubernetes.io/projected/f7933497-5e6f-44d3-9983-17aa99d2a22c-kube-api-access-lzb45\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554539 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554503    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-lib-modules\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554539 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-sys\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554716 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554544    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-proc\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554716 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554600    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-proc\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554716 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554599    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-podres\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.554716 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.554651    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7933497-5e6f-44d3-9983-17aa99d2a22c-lib-modules\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.563841 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.563815    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzb45\" (UniqueName: \"kubernetes.io/projected/f7933497-5e6f-44d3-9983-17aa99d2a22c-kube-api-access-lzb45\") pod \"perf-node-gather-daemonset-tjhh9\" (UID: \"f7933497-5e6f-44d3-9983-17aa99d2a22c\") " pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.680856 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.680836    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:26.808811 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:26.808782    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"]
Jul 22 21:47:26.810829 ip-10-0-131-38 kubenswrapper[2559]: W0722 21:47:26.810806    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7933497_5e6f_44d3_9983_17aa99d2a22c.slice/crio-20e8a1ac816da0ed1137ff6e551e0ffbdd938ac95c4ca25c159f401fd02b5096 WatchSource:0}: Error finding container 20e8a1ac816da0ed1137ff6e551e0ffbdd938ac95c4ca25c159f401fd02b5096: Status 404 returned error can't find the container with id 20e8a1ac816da0ed1137ff6e551e0ffbdd938ac95c4ca25c159f401fd02b5096
Jul 22 21:47:27.521762 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:27.521722    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9" event={"ID":"f7933497-5e6f-44d3-9983-17aa99d2a22c","Type":"ContainerStarted","Data":"3b3d08e1e7c8c96b3b99de68d7d04cccd495df38457ddf04e66da4b5e5374669"}
Jul 22 21:47:27.522146 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:27.521772    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9" event={"ID":"f7933497-5e6f-44d3-9983-17aa99d2a22c","Type":"ContainerStarted","Data":"20e8a1ac816da0ed1137ff6e551e0ffbdd938ac95c4ca25c159f401fd02b5096"}
Jul 22 21:47:27.522146 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:27.521853    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:27.539330 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:27.539284    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9" podStartSLOduration=1.539271255 podStartE2EDuration="1.539271255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 21:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 21:47:27.537812537 +0000 UTC m=+3029.938614545" watchObservedRunningTime="2026-07-22 21:47:27.539271255 +0000 UTC m=+3029.940073237"
Jul 22 21:47:28.195521 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:28.195496    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-tnwdm_ffff2ea6-b0d9-4adc-8b92-ce54ca376455/volume-data-source-validator/0.log"
Jul 22 21:47:29.294294 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:29.294268    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cw4qm_10d8d735-5141-43ff-a8b9-6a35615d62b5/dns/0.log"
Jul 22 21:47:29.313888 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:29.313866    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cw4qm_10d8d735-5141-43ff-a8b9-6a35615d62b5/kube-rbac-proxy/0.log"
Jul 22 21:47:29.384919 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:29.384896    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7cqsd_a0b7daf0-ad29-4a64-916c-959616657b02/dns-node-resolver/0.log"
Jul 22 21:47:29.967321 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:29.967280    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-856988977b-txn6q_726d5fd7-3672-46ad-b3a4-fa2f3d8d40b9/registry/0.log"
Jul 22 21:47:30.034225 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:30.034203    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g2qc8_2e752280-465f-4e6a-83d0-52251da4bc17/node-ca/0.log"
Jul 22 21:47:31.062877 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:31.062843    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pxxm_f6d7a11e-a9a1-4103-97d4-685fc52f8434/discovery/0.log"
Jul 22 21:47:31.114044 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:31.114021    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6766dd8-th24k_b18e2476-a2d3-4c30-bf0b-464aafab41f6/kube-auth-proxy/0.log"
Jul 22 21:47:31.207074 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:31.207051    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-r4486_401d4b9c-7c49-4975-a310-aa2c958a7a5f/payload-pre-processing/0.log"
Jul 22 21:47:31.234224 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:31.234207    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hbp5n_0cbce6c2-b781-4112-b55e-ca19806a301f/payload-processing/0.log"
Jul 22 21:47:31.982695 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:31.982670    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mqn7j_ac9cb0ea-7a64-428e-b599-b13b45e763ce/serve-healthcheck-canary/0.log"
Jul 22 21:47:32.586505 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:32.586476    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dmv75_4e9c6314-dd34-4faa-abfb-e78a50ab275b/kube-rbac-proxy/0.log"
Jul 22 21:47:32.611137 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:32.611112    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dmv75_4e9c6314-dd34-4faa-abfb-e78a50ab275b/exporter/0.log"
Jul 22 21:47:32.633797 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:32.633778    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dmv75_4e9c6314-dd34-4faa-abfb-e78a50ab275b/extractor/0.log"
Jul 22 21:47:33.536386 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:33.536361    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rv7sd/perf-node-gather-daemonset-tjhh9"
Jul 22 21:47:35.018324 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:35.018295    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ss7nx_6bf69893-ad00-45fa-9c76-a6b5dfffc588/manager/0.log"
Jul 22 21:47:35.131052 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:35.131029    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d5898b76-64r57_f6f7acab-dc3f-4540-bb5f-7745ee4c6fdc/manager/0.log"
Jul 22 21:47:35.149765 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:35.149748    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lbs79_26b03ead-8ee3-4bbc-bc12-0124700f9a9d/server/0.log"
Jul 22 21:47:35.216673 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:35.216653    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fzq8x_786ef4eb-1d55-4cdc-af35-895719287f5e/manager/0.log"
Jul 22 21:47:35.294020 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:35.293959    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846644596-wddxk_c03d398d-e157-45a0-892d-cca57f13c587/manager/0.log"
Jul 22 21:47:35.313738 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:35.313723    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-846644596-wddxk_c03d398d-e157-45a0-892d-cca57f13c587/copy-manifests/0.log"
Jul 22 21:47:37.732749 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:37.732718    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pxxm_f6d7a11e-a9a1-4103-97d4-685fc52f8434/discovery/0.log"
Jul 22 21:47:37.760028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:37.760006    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6766dd8-th24k_b18e2476-a2d3-4c30-bf0b-464aafab41f6/kube-auth-proxy/0.log"
Jul 22 21:47:37.843624 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:37.843598    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-r4486_401d4b9c-7c49-4975-a310-aa2c958a7a5f/payload-pre-processing/0.log"
Jul 22 21:47:37.865275 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:37.865244    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hbp5n_0cbce6c2-b781-4112-b55e-ca19806a301f/payload-processing/0.log"
Jul 22 21:47:39.197565 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:39.197532    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pxxm_f6d7a11e-a9a1-4103-97d4-685fc52f8434/discovery/0.log"
Jul 22 21:47:39.230515 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:39.230488    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6766dd8-th24k_b18e2476-a2d3-4c30-bf0b-464aafab41f6/kube-auth-proxy/0.log"
Jul 22 21:47:39.313520 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:39.313498    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-r4486_401d4b9c-7c49-4975-a310-aa2c958a7a5f/payload-pre-processing/0.log"
Jul 22 21:47:39.332610 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:39.332589    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hbp5n_0cbce6c2-b781-4112-b55e-ca19806a301f/payload-processing/0.log"
Jul 22 21:47:41.958072 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:41.958044    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-479sj_c713457b-8861-4ce3-97c8-d6a8d6c73670/migrator/0.log"
Jul 22 21:47:41.979188 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:41.979165    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-479sj_c713457b-8861-4ce3-97c8-d6a8d6c73670/graceful-termination/0.log"
Jul 22 21:47:43.575583 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.575558    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxtv6_4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5/kube-multus-additional-cni-plugins/0.log"
Jul 22 21:47:43.596185 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.596160    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxtv6_4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5/egress-router-binary-copy/0.log"
Jul 22 21:47:43.616613 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.616574    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxtv6_4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5/cni-plugins/0.log"
Jul 22 21:47:43.636305 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.636285    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxtv6_4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5/bond-cni-plugin/0.log"
Jul 22 21:47:43.655112 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.655092    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxtv6_4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5/routeoverride-cni/0.log"
Jul 22 21:47:43.676744 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.676727    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxtv6_4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5/whereabouts-cni-bincopy/0.log"
Jul 22 21:47:43.696445 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.696428    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xxtv6_4a30d582-b3cb-43a0-b8e3-a8ffed74cbb5/whereabouts-cni/0.log"
Jul 22 21:47:43.759052 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.759032    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wk5qr_77de0270-f368-45b6-8b34-b26c6febbb3a/kube-multus/0.log"
Jul 22 21:47:43.878028 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.877976    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b62h4_4df97b04-40ba-466d-a735-cd1184db9185/network-metrics-daemon/0.log"
Jul 22 21:47:43.893345 ip-10-0-131-38 kubenswrapper[2559]: I0722 21:47:43.893328    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b62h4_4df97b04-40ba-466d-a735-cd1184db9185/kube-rbac-proxy/0.log"