Jul 16 22:14:58.974328 ip-10-0-139-105 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 22:14:58.974341 ip-10-0-139-105 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 22:14:58.974348 ip-10-0-139-105 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 22:14:58.974750 ip-10-0-139-105 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 22:15:09.040577 ip-10-0-139-105 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 22:15:09.040605 ip-10-0-139-105 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 07f051bc55774801b2e3df0bf9a76195 --
Jul 16 22:16:45.738199 ip-10-0-139-105 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 22:16:46.133907 ip-10-0-139-105 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:16:46.133907 ip-10-0-139-105 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 22:16:46.133907 ip-10-0-139-105 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:16:46.133907 ip-10-0-139-105 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 22:16:46.133907 ip-10-0-139-105 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:16:46.134620 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.133999    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 22:16:46.141907 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141877    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:16:46.141907 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141906    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:16:46.141907 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141911    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141914    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141917    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141921    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141924    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141930    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141940    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141942    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141945    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141948    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141951    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141953    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141956    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141958    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141961    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141964    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141966    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141969    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141971    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:16:46.142009 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141974    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141976    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141980    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141982    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141985    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141987    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141990    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141992    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141995    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.141997    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142000    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142002    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142005    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142007    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142009    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142013    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142018    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142023    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142028    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:16:46.142449 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142031    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142034    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142037    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142039    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142042    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142044    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142048    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142050    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142053    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142055    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142058    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142060    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142063    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142065    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142067    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142070    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142072    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142075    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142077    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142080    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:16:46.142920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142082    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142084    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142087    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142089    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142092    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142095    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142097    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142100    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142103    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142106    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142108    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142112    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142114    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142117    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142119    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142121    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142124    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142126    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142129    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142131    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:16:46.143408 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142134    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142136    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142138    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142141    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142143    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142146    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142148    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142150    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142153    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142155    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142157    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142160    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142162    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142164    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142167    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142169    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142172    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142174    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142178    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142877    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:16:46.143939 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142884    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142887    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142890    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142908    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142912    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142915    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142917    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142920    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142922    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142925    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142927    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142930    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142932    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142935    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142937    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142940    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142942    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142945    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142947    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142950    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:16:46.144418 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142952    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142955    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142958    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142960    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142962    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142965    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142967    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142970    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142973    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142975    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142978    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142981    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142986    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142990    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142993    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.142996    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143000    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143004    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143007    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:16:46.144920 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143010    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143013    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143015    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143018    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143020    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143023    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143025    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143028    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143030    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143033    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143035    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143037    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143040    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143043    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143046    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143048    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143051    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143053    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143055    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143058    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:16:46.145396 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143060    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143063    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143065    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143068    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143071    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143074    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143076    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143079    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143081    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143083    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143086    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143089    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143091    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143094    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143097    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143101    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143103    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143106    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143108    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143111    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:16:46.145875 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143113    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143116    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143118    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143120    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143123    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143125    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143128    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143130    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143133    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143135    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143138    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143140    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143143    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143145    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143148    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143151    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143153    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143156    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143159    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143531    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:16:46.146372 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143536    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143538    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143541    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143545    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143549    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143553    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143555    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143558    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143562    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143566    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143569    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143572    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143574    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143577    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143580    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143583    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143585    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143588    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143591    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:16:46.146847 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143593    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143596    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143599    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143601    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143604    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143606    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143609    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143611    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143614    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143616    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143620    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143622    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143625    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143627    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143630    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143632    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143634    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143638    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143640    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143642    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:16:46.147309 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143645    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143647    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143650    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143653    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143655    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143657    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143660    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143662    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143665    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143668    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143670    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143673    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143675    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143677    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143680    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143683    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143685    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143687    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143690    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143692    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:16:46.147792 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143695    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143697    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143700    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143703    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143705    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143707    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143710    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143712    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143714    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143717    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143720    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143723    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143725    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143728    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143730    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143734    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143737    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143739    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143743    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:16:46.148313 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143746    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143748    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143751    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143754    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143756    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143759    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143761    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143764    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143767    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143769    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143772    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143774    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143776    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143779    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143782    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143784    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143787    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143789    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143792    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.143794    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:16:46.148819 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143875    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143886    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143920    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143927    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143934    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143939    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143944    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143949    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143952    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143955    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143959    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143962    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143965    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143969    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143971    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143974    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143977    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143980    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143984    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143992    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143995    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.143998    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144001    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144005    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 22:16:46.149317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144008    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144011    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144015    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144018    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144021    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144024    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144027    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144030    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144034    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144037    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144040    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144042    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144046    2556 flags.go:64] FLAG: --enable-server="true"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144049    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144054    2556 flags.go:64] FLAG: --event-burst="100"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144057    2556 flags.go:64] FLAG: --event-qps="50"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144060    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144063    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144069    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144073    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144076    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144079    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144082    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144085    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144091    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 22:16:46.149908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144094    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144097    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144099    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144102    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144105    2556 flags.go:64] FLAG: --feature-gates=""
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144109    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144112    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144115    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144118    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144121    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144124    2556 flags.go:64] FLAG: --help="false"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144135    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144139    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144142    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144145    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144149    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144152    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144155    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144157    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144160    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144163    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144167    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144170    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144172    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 16 22:16:46.150530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144175    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144179    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144182    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144185    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144188    2556 flags.go:64] FLAG: --lock-file=""
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144190    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144193    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144197    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144203    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144206    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144209    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144211    2556 flags.go:64] FLAG: --logging-format="text"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144214    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144217    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144220    2556 flags.go:64] FLAG: --manifest-url=""
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144223    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144227    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144230    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144234    2556 flags.go:64] FLAG: --max-pods="110"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144237    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144240    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144243    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144246    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144249    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144252    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 22:16:46.151162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144255    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144262    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144265    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144268    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144271    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144275    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144280    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144284    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144291    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144294    2556 flags.go:64] FLAG: --port="10250"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144297    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144300    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00f6cf860f2f22c3f"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144303    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144306    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144308    2556 flags.go:64] FLAG: --register-node="true"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144312    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144316    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144319    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144322    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144324    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144328    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144331    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144334    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144337    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144339    2556 flags.go:64] FLAG: --runonce="false"
Jul 16 22:16:46.151796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144342    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144345    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144348    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144351    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144354    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144357    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144360    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144362    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144365    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144368    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144371    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144373    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144376    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144380    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144382    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144390    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144393    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144395    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144400    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144402    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144405    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144408    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144412    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144415    2556 flags.go:64] FLAG: --v="2"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144424    2556 flags.go:64] FLAG: --version="false"
Jul 16 22:16:46.152428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144428    2556 flags.go:64] FLAG: --vmodule=""
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144432    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.144436    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144571    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144575    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144580    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144584    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144587    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144590    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144593    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144596    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144598    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144601    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144604    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144606    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144609    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144611    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144614    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144617    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144619    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:16:46.153041 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144621    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144625    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144629    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144631    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144634    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144637    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144639    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144641    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144644    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144647    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144650    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144652    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144655    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144658    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144660    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144662    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144665    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144667    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144670    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144672    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:16:46.153520 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144674    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144677    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144679    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144682    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144684    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144686    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144689    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144691    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144694    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144696    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144699    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144701    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144704    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144706    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144710    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144712    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144715    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144717    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144720    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:16:46.154021 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144722    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144724    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144728    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144731    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144735    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144738    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144741    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144744    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144747    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144749    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144751    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144754    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144758    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144761    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144764    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144767    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144769    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144771    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144774    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:16:46.154484 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144776    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144778    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144781    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144783    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144786    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144789    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144791    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144794    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144798    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144801    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144803    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144806    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144808    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144811    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144813    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144816    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144819    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144821    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144823    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144826    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:16:46.154966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144828    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144831    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144833    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.144836    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.145440    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.152205    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.152223    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.152246    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.152254    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152311    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152316    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152320    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152324    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152327    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152330    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:16:46.155572 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152332    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152335    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152337    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152340    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152342    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152345    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152347    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152350    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152353    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152356    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152358    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152360    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152363    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152365    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152368    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152370    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152373    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152376    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152378    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152381    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:16:46.156252 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152383    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152386    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152388    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152393    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152396    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152398    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152401    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152404    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152406    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152409    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152411    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152414    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152416    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152418    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152421    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152423    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152425    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152428    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152430    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152433    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:16:46.156750 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152437    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152440    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152443    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152446    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152448    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152451    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152453    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152456    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152458    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152461    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152463    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152466    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152469    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152471    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152474    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152477    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152480    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152482    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152485    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152488    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:16:46.157285 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152492    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152496    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152499    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152502    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152505    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152507    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152510    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152513    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152516    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152519    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152521    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152524    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152527    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152530    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152534    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152537    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152539    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152542    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152545    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:16:46.157764 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152547    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152550    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152552    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152555    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152558    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152560    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152563    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152565    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152568    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152571    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152574    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152576    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152579    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152581    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.152587    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:16:46.158461 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152699    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152705    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152708    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152711    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152713    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152716    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152718    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152721    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152723    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152726    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152728    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152731    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152734    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152736    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152738    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152741    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152743    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152746    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152749    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152751    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:16:46.158848 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152753    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152756    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152758    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152761    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152763    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152765    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152768    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152771    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152774    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152776    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152778    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152781    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152783    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152786    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152788    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152790    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152793    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152795    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152797    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152800    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:16:46.159373 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152802    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152806    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152810    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152812    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152816    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152819    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152822    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152825    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152827    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152830    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152832    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152835    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152838    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152840    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152843    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152846    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152849    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152852    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152855    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:16:46.159929 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152858    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152861    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152864    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152867    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152869    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152872    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152874    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152876    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152879    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152881    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152883    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152887    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152890    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152907    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152910    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152912    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152915    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152917    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152920    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:16:46.160539 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152922    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152925    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152927    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152929    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152932    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152934    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152937    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152939    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152942    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152944    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152947    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152949    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152952    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152955    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152958    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152960    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152963    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152965    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152968    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152970    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:16:46.161010 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:46.152973    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:16:46.161500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.152978    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:16:46.161500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.153102    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 22:16:46.161500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.155117    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 22:16:46.161500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.159982    2556 server.go:1019] "Starting client certificate rotation"
Jul 16 22:16:46.161500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.160095    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 22:16:46.161500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.160151    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 22:16:46.162450 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.162434    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 22:16:46.163571 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.163555    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 22:16:46.175197 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.175178    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 16 22:16:46.177697 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.177680    2556 log.go:25] "Validated CRI v1 image API"
Jul 16 22:16:46.178497 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.178481    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 22:16:46.183552 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.183530    2556 fs.go:135] Filesystem UUIDs: map[209b3a90-1040-4943-9497-3e86d669c5bf:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 faaf9eca-53cf-48e6-a950-5ffca3da26f0:/dev/nvme0n1p4]
Jul 16 22:16:46.183637 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.183552    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 22:16:46.188544 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.188428    2556 manager.go:216] Machine: {Timestamp:2026-07-16 22:16:46.187255156 +0000 UTC m=+0.336730104 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3037754 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec252ff7373cf4e1d85e3fb10dad4d8c SystemUUID:ec252ff7-373c-f4e1-d85e-3fb10dad4d8c BootID:07f051bc-5577-4801-b2e3-df0bf9a76195 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:87:ec:f6:62:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:87:ec:f6:62:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:2e:bd:54:1f:9c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 22:16:46.188544 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.188531    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 22:16:46.188708 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.188638    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 22:16:46.189501 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.189485    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 22:16:46.191810 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.191780    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 22:16:46.191966 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.191815    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 22:16:46.193009 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.192999    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 22:16:46.193043 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.193012    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 22:16:46.193043 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.193020    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 22:16:46.193593 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.193583    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 22:16:46.194809 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.194799    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 22:16:46.198014 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198003    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 22:16:46.198175 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198165    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 22:16:46.198206 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198200    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 22:16:46.198234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198212    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 22:16:46.198234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198227    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 22:16:46.198288 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198236    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 16 22:16:46.198288 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198245    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 22:16:46.198600 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198587    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 22:16:46.198656 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.198606    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 22:16:46.203374 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.203347    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 22:16:46.205482 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.205468    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 22:16:46.205538 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.205499    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 22:16:46.206608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206596    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 22:16:46.206651 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206614    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 22:16:46.206651 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206620    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 22:16:46.206651 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206626    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 22:16:46.206651 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206635    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 22:16:46.206651 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206641    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 22:16:46.206651 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206649    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 22:16:46.206842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206660    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 22:16:46.206842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206667    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 22:16:46.206842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206672    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 22:16:46.206842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206680    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 22:16:46.206842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.206685    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 22:16:46.207809 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.207790    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 22:16:46.207809 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.207808    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 22:16:46.209709 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.209687    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 22:16:46.209810 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.209783    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 22:16:46.209810 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.209791    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 22:16:46.211079 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.211063    2556 server.go:1268] "Started kubelet"
Jul 16 22:16:46.211186 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.211140    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 22:16:46.211281 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.211167    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 22:16:46.211281 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.211255    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 22:16:46.212027 ip-10-0-139-105 systemd[1]: Started Kubernetes Kubelet.
Jul 16 22:16:46.213405 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.213392    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 22:16:46.215997 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.215979    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 22:16:46.217388 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.217359    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 22:16:46.221657 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.221638    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 22:16:46.221745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.221643    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 22:16:46.221801 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.221738    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.222225 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222206    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 22:16:46.222225 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222218    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 22:16:46.222361 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222233    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 22:16:46.222361 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222346    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 22:16:46.222361 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222356    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 22:16:46.222613 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222596    2556 factory.go:55] Registering systemd factory
Jul 16 22:16:46.222669 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222636    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 16 22:16:46.222847 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222832    2556 factory.go:153] Registering CRI-O factory
Jul 16 22:16:46.222847 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222849    2556 factory.go:223] Registration of the crio container factory successfully
Jul 16 22:16:46.222967 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222911    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 22:16:46.222967 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222935    2556 factory.go:103] Registering Raw factory
Jul 16 22:16:46.222967 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.222949    2556 manager.go:1196] Started watching for new ooms in manager
Jul 16 22:16:46.223679 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.223662    2556 manager.go:349] Starting recovery of all containers
Jul 16 22:16:46.225206 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.225177    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 22:16:46.225469 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.225420    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 22:16:46.226157 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.225157    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-105.ec2.internal.18c2e503a52db8e6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-105.ec2.internal,UID:ip-10-0-139-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-105.ec2.internal,},FirstTimestamp:2026-07-16 22:16:46.211037414 +0000 UTC m=+0.360512350,LastTimestamp:2026-07-16 22:16:46.211037414 +0000 UTC m=+0.360512350,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-105.ec2.internal,}"
Jul 16 22:16:46.234436 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.234413    2556 manager.go:354] Recovery completed
Jul 16 22:16:46.236241 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.236100    2556 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 16 22:16:46.239581 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.239566    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:16:46.241595 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.241572    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:16:46.241655 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.241608    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:16:46.241655 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.241618    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:16:46.242158 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.242144    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 22:16:46.242158 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.242157    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 22:16:46.242249 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.242172    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 22:16:46.242249 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.242192    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:16:46.242830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.242815    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:16:46.242877 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.242840    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:16:46.242877 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.242850    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:16:46.243419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.243390    2556 policy_none.go:49] "None policy: Start"
Jul 16 22:16:46.243462 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.243423    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 22:16:46.243462 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.243433    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 22:16:46.243738 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.243668    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-105.ec2.internal.18c2e503a70000d2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-105.ec2.internal,UID:ip-10-0-139-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-105.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-105.ec2.internal,},FirstTimestamp:2026-07-16 22:16:46.241595602 +0000 UTC m=+0.391070538,LastTimestamp:2026-07-16 22:16:46.241595602 +0000 UTC m=+0.391070538,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-105.ec2.internal,}"
Jul 16 22:16:46.244520 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.244508    2556 policy_none.go:47] "Start"
Jul 16 22:16:46.245982 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.245967    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mw5hz"
Jul 16 22:16:46.252990 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.252931    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-105.ec2.internal.18c2e503a700446d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-105.ec2.internal,UID:ip-10-0-139-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-105.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-105.ec2.internal,},FirstTimestamp:2026-07-16 22:16:46.241612909 +0000 UTC m=+0.391087845,LastTimestamp:2026-07-16 22:16:46.241612909 +0000 UTC m=+0.391087845,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-105.ec2.internal,}"
Jul 16 22:16:46.253829 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.253815    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mw5hz"
Jul 16 22:16:46.280516 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.280500    2556 manager.go:337] "Starting Device Plugin manager"
Jul 16 22:16:46.280636 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.280531    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 22:16:46.280636 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.280541    2556 server.go:87] "Starting device plugin registration server"
Jul 16 22:16:46.280786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.280773    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 22:16:46.281046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.280788    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 22:16:46.281046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.280971    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 22:16:46.281046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.281038    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 22:16:46.281046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.281046    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 22:16:46.281907 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.281615    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 22:16:46.281907 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.281671    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.327397 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.327355    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 22:16:46.328745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.328723    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 22:16:46.328814 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.328763    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 22:16:46.328814 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.328792    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 22:16:46.328921 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.328821    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 22:16:46.331581 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.331563    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:16:46.381922 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.381880    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:16:46.383052 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.383031    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:16:46.383154 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.383062    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:16:46.383154 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.383072    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:16:46.383154 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.383096    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.392733 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.392688    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.392733 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.392710    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-105.ec2.internal\": node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.421199 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.421178    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.429533 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.429516    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"]
Jul 16 22:16:46.429581 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.429567    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:16:46.430351 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.430334    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:16:46.430422 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.430366    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:16:46.430422 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.430383    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:16:46.431790 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.431777    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:16:46.431954 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.431938    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.431993 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.431984    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:16:46.432419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.432399    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:16:46.432506 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.432429    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:16:46.432506 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.432444    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:16:46.432621 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.432603    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:16:46.432701 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.432629    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:16:46.432701 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.432639    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:16:46.433645 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.433629    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.433715 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.433656    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:16:46.434329 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.434312    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:16:46.434384 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.434336    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:16:46.434384 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.434350    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:16:46.461694 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.461671    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-105.ec2.internal\" not found" node="ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.466002 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.465987    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-105.ec2.internal\" not found" node="ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.521240 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.521215    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.523455 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.523438    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a88942796a5b6b2c5f36a7e709e10120-config\") pod \"kube-apiserver-proxy-ip-10-0-139-105.ec2.internal\" (UID: \"a88942796a5b6b2c5f36a7e709e10120\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.523532 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.523469    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a91334f2b2daa9817292f0040684ba7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal\" (UID: \"6a91334f2b2daa9817292f0040684ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.523532 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.523509    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a91334f2b2daa9817292f0040684ba7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal\" (UID: \"6a91334f2b2daa9817292f0040684ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.621720 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.621675    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.623884 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.623870    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a88942796a5b6b2c5f36a7e709e10120-config\") pod \"kube-apiserver-proxy-ip-10-0-139-105.ec2.internal\" (UID: \"a88942796a5b6b2c5f36a7e709e10120\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.623956 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.623904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a91334f2b2daa9817292f0040684ba7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal\" (UID: \"6a91334f2b2daa9817292f0040684ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.623956 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.623928    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a91334f2b2daa9817292f0040684ba7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal\" (UID: \"6a91334f2b2daa9817292f0040684ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.623956 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.623954    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a91334f2b2daa9817292f0040684ba7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal\" (UID: \"6a91334f2b2daa9817292f0040684ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.624078 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.623982    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a91334f2b2daa9817292f0040684ba7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal\" (UID: \"6a91334f2b2daa9817292f0040684ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.624078 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.623980    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a88942796a5b6b2c5f36a7e709e10120-config\") pod \"kube-apiserver-proxy-ip-10-0-139-105.ec2.internal\" (UID: \"a88942796a5b6b2c5f36a7e709e10120\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.722434 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.722368    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.764729 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.764708    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.769673 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:46.769652    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:46.823478 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.823449    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:46.923941 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:46.923878    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:47.024428 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.024361    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:47.094256 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.094225    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:16:47.124699 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.124666    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-105.ec2.internal\" not found"
Jul 16 22:16:47.143615 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.143586    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:16:47.158799 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.158779    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 22:16:47.158945 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.158925    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 22:16:47.158988 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.158939    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 22:16:47.158988 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.158962    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 22:16:47.199124 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.199083    2556 apiserver.go:52] "Watching apiserver"
Jul 16 22:16:47.217058 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.217022    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 22:16:47.217358 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.217336    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-fjq4d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg","openshift-cluster-node-tuning-operator/tuned-85pdd","openshift-dns/node-resolver-7nft5","openshift-multus/multus-additional-cni-plugins-cptpd","openshift-network-diagnostics/network-check-target-52pn4","openshift-network-operator/iptables-alerter-28mf2","openshift-ovn-kubernetes/ovnkube-node-8k4z4","kube-system/global-pull-secret-syncer-hhz6k","openshift-image-registry/node-ca-8gvtz","openshift-multus/multus-dbspd","openshift-multus/network-metrics-daemon-fd92h"]
Jul 16 22:16:47.218866 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.218849    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.220435 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.220417    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.220784 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.220763    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 22:16:47.221086 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.221069    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lqk6p\""
Jul 16 22:16:47.221131 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.221083    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 22:16:47.221162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.221078    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 22:16:47.221513 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.221501    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.221774 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.221754    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 22:16:47.221832 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.221809    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:47.222603 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.222581    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:47.222715 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.222649    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 22:16:47.222715 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.222680    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:16:47.222715 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.222691    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 22:16:47.222715 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.222703    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 22:16:47.222925 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.222796    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 22:16:47.223024 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.223010    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fj487\""
Jul 16 22:16:47.223130 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.223117    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 22:16:47.223767 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.223750    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.224071 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.224051    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:16:47.224173 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.224053    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 22:16:47.224173 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.224145    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 22:16:47.224287 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.224151    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gk8jc\""
Jul 16 22:16:47.224973 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.224956    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.225790 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.225775    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 22:16:47.225877 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.225792    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gph8l\""
Jul 16 22:16:47.226144 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226119    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.226320 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226305    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 22:16:47.226872 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226853    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.226971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226881    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdvlm\" (UniqueName: \"kubernetes.io/projected/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-kube-api-access-rdvlm\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.226971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226913    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ds6kb\""
Jul 16 22:16:47.226971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226915    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bbhm\" (UniqueName: \"kubernetes.io/projected/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-kube-api-access-5bbhm\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.226971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226942    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:16:47.227169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.226978    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrdc2\" (UniqueName: \"kubernetes.io/projected/b45c42e2-acce-454f-8aa5-59e0b573e914-kube-api-access-nrdc2\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:47.227169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-os-release\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.227169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227051    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-iptables-alerter-script\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.227169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.227169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227096    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-socket-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.227169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227112    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-sys-fs\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.227169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227128    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-system-cni-dir\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227174    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227190    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227208    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227221    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-registration-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227247    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-device-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227273    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-etc-selinux\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227296    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cni-binary-copy\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227322    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.227372    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227319    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227405    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-host-slash\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227421    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jffjr\" (UniqueName: \"kubernetes.io/projected/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-kube-api-access-jffjr\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.227444 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.227436    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cnibin\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.228590 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.228578    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.229030 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.229011    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zxtpw\""
Jul 16 22:16:47.229161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.229146    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 22:16:47.229658 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.229643    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 22:16:47.229765 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.229751    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.229836 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.229818    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:16:47.231019 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.231007    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.232266 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.232250    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.233406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233046    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 22:16:47.233406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233079    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 22:16:47.233406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233100    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jd654\""
Jul 16 22:16:47.233406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233176    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 22:16:47.233406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233208    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 22:16:47.233406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233271    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 22:16:47.233406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233342    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 22:16:47.234143 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233451    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 22:16:47.234143 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233535    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 22:16:47.234143 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.233547    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j22mb\""
Jul 16 22:16:47.234280 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.234168    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 22:16:47.234461 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.234444    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jrp5w\""
Jul 16 22:16:47.234518 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.234505    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 22:16:47.234987 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.234972    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 22:16:47.241762 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.241388    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 22:16:47.241762 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.241667    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal"]
Jul 16 22:16:47.243243 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.242049    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal"
Jul 16 22:16:47.254562 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.254543    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 22:16:47.254656 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.254571    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal"]
Jul 16 22:16:47.257790 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.257739    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 22:11:47 +0000 UTC" deadline="2028-04-20 04:06:02.82030023 +0000 UTC"
Jul 16 22:16:47.257790 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.257786    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15437h49m15.562516761s"
Jul 16 22:16:47.259000 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.258982    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r6685"
Jul 16 22:16:47.267101 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.267085    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r6685"
Jul 16 22:16:47.323306 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.323285    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 22:16:47.327690 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327660    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:47.327813 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327709    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovnkube-config\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.327813 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327747    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8ld8\" (UniqueName: \"kubernetes.io/projected/622ef6b4-0643-4162-b807-27de4bcd3ffd-kube-api-access-n8ld8\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.327813 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327772    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-conf-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.327813 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327796    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97d954b6-fe98-486b-a51e-a3020d766739-agent-certs\") pod \"konnectivity-agent-fjq4d\" (UID: \"97d954b6-fe98-486b-a51e-a3020d766739\") " pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327828    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-registration-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327858    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cni-binary-copy\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327884    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327926    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4d9b940-f340-4329-aea0-6252d6d331d0-kubelet-config\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327942    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-registration-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327955    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjq24\" (UniqueName: \"kubernetes.io/projected/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-kube-api-access-fjq24\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.327980    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-kubernetes\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.328032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-systemd\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-run\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328069    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-node-log\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328094    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-cni-bin\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328163    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-cni-multus\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328198    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysctl-d\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328221    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-hostroot\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328270    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328293    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-slash\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328333    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysctl-conf\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.328413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrdc2\" (UniqueName: \"kubernetes.io/projected/b45c42e2-acce-454f-8aa5-59e0b573e914-kube-api-access-nrdc2\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328427    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-var-lib-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328454    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328471    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cni-binary-copy\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328479    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovn-node-metrics-cert\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328503    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jffjr\" (UniqueName: \"kubernetes.io/projected/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-kube-api-access-jffjr\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328530    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328556    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdvlm\" (UniqueName: \"kubernetes.io/projected/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-kube-api-access-rdvlm\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328580    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bbhm\" (UniqueName: \"kubernetes.io/projected/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-kube-api-access-5bbhm\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328601    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-run-netns\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328624    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1226feab-5f42-4977-af53-d9d2957bd99e-cni-binary-copy\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328652    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-k8s-cni-cncf-io\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328688    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-hosts-file\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328712    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-iptables-alerter-script\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328740    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-tuned\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328761    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db232de-e5b6-4cb6-99e0-32fa465d7003-tmp\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.328978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328783    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-socket-dir-parent\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.329617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328802    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-etc-kubernetes\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.329617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328831    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlbb4\" (UniqueName: \"kubernetes.io/projected/1226feab-5f42-4977-af53-d9d2957bd99e-kube-api-access-qlbb4\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.329617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328862    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxdwk\" (UniqueName: \"kubernetes.io/projected/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-kube-api-access-cxdwk\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.329617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.329431    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.329617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.329437    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.329984 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.329963    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-iptables-alerter-script\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.330229 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.328886    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.330282 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330254    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-system-cni-dir\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.330332 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-lib-modules\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.330332 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330304    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-var-lib-kubelet\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.330332 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330325    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-host\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.330454 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330347    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-etc-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.330454 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330370    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-log-socket\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.330454 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330396    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-device-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.330547 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330454    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-etc-selinux\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.330547 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330484    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysconfig\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.330547 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330512    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-cni-bin\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.330547 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330539    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-kubelet\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.330713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330564    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-tmp-dir\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.330713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330589    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59hgb\" (UniqueName: \"kubernetes.io/projected/2db232de-e5b6-4cb6-99e0-32fa465d7003-kube-api-access-59hgb\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.330713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330653    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-systemd\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.330713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330677    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-env-overrides\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.330852 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330717    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-serviceca\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.330852 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330743    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-os-release\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.330852 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-modprobe-d\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.330852 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-systemd-units\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.330852 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330823    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-ovn\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.331020 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330853    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.331020 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330847    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovnkube-script-lib\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.331020 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330957    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-system-cni-dir\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.331020 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.330963    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-device-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.331133 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331036    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-etc-selinux\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.331133 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331088    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-os-release\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.331133 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331114    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-multus-certs\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.331234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331137    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-socket-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.331234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331154    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-sys-fs\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.331234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331182    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:47.331234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331211    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-sys-fs\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.331234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331222    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-run-ovn-kubernetes\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331244    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4d9b940-f340-4329-aea0-6252d6d331d0-dbus\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331271    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-socket-dir\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331300    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-cni-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.331322    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331332    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331357    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-host-slash\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.331390    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs podName:b45c42e2-acce-454f-8aa5-59e0b573e914 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:47.831362058 +0000 UTC m=+1.980836986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs") pod "network-metrics-daemon-fd92h" (UID: "b45c42e2-acce-454f-8aa5-59e0b573e914") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:47.331425 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331406    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-host-slash\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331444    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-os-release\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-netns\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331479    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97d954b6-fe98-486b-a51e-a3020d766739-konnectivity-ca\") pod \"konnectivity-agent-fjq4d\" (UID: \"97d954b6-fe98-486b-a51e-a3020d766739\") " pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331496    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-host\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cnibin\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331539    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-sys\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331549    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cnibin\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331557    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-kubelet\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331581    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-cni-netd\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331605    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-system-cni-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331621    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-cnibin\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.331795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331637    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1226feab-5f42-4977-af53-d9d2957bd99e-multus-daemon-config\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.332315 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.331863    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.335398 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.335378    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 22:16:47.338717 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.338695    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdvlm\" (UniqueName: \"kubernetes.io/projected/4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584-kube-api-access-rdvlm\") pod \"multus-additional-cni-plugins-cptpd\" (UID: \"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584\") " pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.338802 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.338695    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrdc2\" (UniqueName: \"kubernetes.io/projected/b45c42e2-acce-454f-8aa5-59e0b573e914-kube-api-access-nrdc2\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:47.338802 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.338751    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jffjr\" (UniqueName: \"kubernetes.io/projected/ad8c3fb4-78f2-4f65-a43c-86afbe4cc136-kube-api-access-jffjr\") pod \"aws-ebs-csi-driver-node-4qlhg\" (UID: \"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.338802 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.338751    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bbhm\" (UniqueName: \"kubernetes.io/projected/24ee481c-767b-4a1c-9dd3-57a1e8f56d2b-kube-api-access-5bbhm\") pod \"iptables-alerter-28mf2\" (UID: \"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b\") " pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.418248 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.418216    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88942796a5b6b2c5f36a7e709e10120.slice/crio-56df0a1fcef21f95c4b785bf46e48536ae0ca69a53498c5d6f644c0c98277151 WatchSource:0}: Error finding container 56df0a1fcef21f95c4b785bf46e48536ae0ca69a53498c5d6f644c0c98277151: Status 404 returned error can't find the container with id 56df0a1fcef21f95c4b785bf46e48536ae0ca69a53498c5d6f644c0c98277151
Jul 16 22:16:47.418944 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.418923    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a91334f2b2daa9817292f0040684ba7.slice/crio-c11159515a4706f8986239a9e2ea2a8b5a505dffab2388c348c4cb019ce07b5f WatchSource:0}: Error finding container c11159515a4706f8986239a9e2ea2a8b5a505dffab2388c348c4cb019ce07b5f: Status 404 returned error can't find the container with id c11159515a4706f8986239a9e2ea2a8b5a505dffab2388c348c4cb019ce07b5f
Jul 16 22:16:47.423118 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.423104    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:16:47.432061 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432039    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-modprobe-d\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.432181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-systemd-units\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.432181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432083    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-ovn\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.432181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432100    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovnkube-script-lib\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.432181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432115    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-multus-certs\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432158    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-systemd-units\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.432181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432173    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-multus-certs\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432188    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-ovn\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432198    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-modprobe-d\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432222    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-run-ovn-kubernetes\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432246    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4d9b940-f340-4329-aea0-6252d6d331d0-dbus\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432265    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-run-ovn-kubernetes\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432268    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-cni-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432310    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-os-release\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432340    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-netns\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432349    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c4d9b940-f340-4329-aea0-6252d6d331d0-dbus\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432357    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97d954b6-fe98-486b-a51e-a3020d766739-konnectivity-ca\") pod \"konnectivity-agent-fjq4d\" (UID: \"97d954b6-fe98-486b-a51e-a3020d766739\") " pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432372    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-cni-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432379    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-host\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432400    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-os-release\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432402    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-netns\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.432408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432412    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-host\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432409    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-sys\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432436    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-kubelet\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432447    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-sys\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432449    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-cni-netd\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432468    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-cni-netd\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432484    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-system-cni-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432494    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-kubelet\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432508    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-cnibin\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432531    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-system-cni-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1226feab-5f42-4977-af53-d9d2957bd99e-multus-daemon-config\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432568    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-cnibin\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432569    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432594    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovnkube-config\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432616    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8ld8\" (UniqueName: \"kubernetes.io/projected/622ef6b4-0643-4162-b807-27de4bcd3ffd-kube-api-access-n8ld8\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432636    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-conf-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432661    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97d954b6-fe98-486b-a51e-a3020d766739-agent-certs\") pod \"konnectivity-agent-fjq4d\" (UID: \"97d954b6-fe98-486b-a51e-a3020d766739\") " pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432699    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.433046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432724    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4d9b940-f340-4329-aea0-6252d6d331d0-kubelet-config\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432726    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovnkube-script-lib\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432749    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjq24\" (UniqueName: \"kubernetes.io/projected/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-kube-api-access-fjq24\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432772    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-kubernetes\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432794    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-systemd\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432814    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-run\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432837    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-node-log\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-conf-dir\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-cni-bin\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432919    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-cni-bin\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432930    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-cni-multus\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432941    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-kubernetes\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97d954b6-fe98-486b-a51e-a3020d766739-konnectivity-ca\") pod \"konnectivity-agent-fjq4d\" (UID: \"97d954b6-fe98-486b-a51e-a3020d766739\") " pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432972    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-systemd\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.432959    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysctl-d\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433010    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-hostroot\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433016    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-cni-multus\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433015    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-run\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.434169 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433018    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c4d9b940-f340-4329-aea0-6252d6d331d0-kubelet-config\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-slash\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-node-log\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433102    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysctl-conf\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1226feab-5f42-4977-af53-d9d2957bd99e-multus-daemon-config\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433130    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-var-lib-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.433138    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433148    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysctl-d\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433154    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433161    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433104    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-hostroot\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433186    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovn-node-metrics-cert\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.433194    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret podName:c4d9b940-f340-4329-aea0-6252d6d331d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:47.933177804 +0000 UTC m=+2.082652738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret") pod "global-pull-secret-syncer-hhz6k" (UID: "c4d9b940-f340-4329-aea0-6252d6d331d0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433207    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-var-lib-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433217    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysctl-conf\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433131    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovnkube-config\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433229    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433241    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-slash\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433270    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-run-netns\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433291    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1226feab-5f42-4977-af53-d9d2957bd99e-cni-binary-copy\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433306    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-k8s-cni-cncf-io\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433319    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-hosts-file\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433327    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-host-run-netns\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433341    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-tuned\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433363    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-run-k8s-cni-cncf-io\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433385    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db232de-e5b6-4cb6-99e0-32fa465d7003-tmp\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433402    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-hosts-file\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433411    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-socket-dir-parent\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433446    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-etc-kubernetes\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433470    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlbb4\" (UniqueName: \"kubernetes.io/projected/1226feab-5f42-4977-af53-d9d2957bd99e-kube-api-access-qlbb4\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-multus-socket-dir-parent\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-etc-kubernetes\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433589    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxdwk\" (UniqueName: \"kubernetes.io/projected/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-kube-api-access-cxdwk\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433709    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-lib-modules\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.435830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-var-lib-kubelet\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433753    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-host\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433776    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-etc-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433785    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1226feab-5f42-4977-af53-d9d2957bd99e-cni-binary-copy\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433799    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-log-socket\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433838    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysconfig\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433840    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-log-socket\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433846    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-lib-modules\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433885    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-etc-openvswitch\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.433965    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-sysconfig\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434002    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-var-lib-kubelet\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434019    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2db232de-e5b6-4cb6-99e0-32fa465d7003-host\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434002    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-cni-bin\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434090    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-kubelet\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434097    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-cni-bin\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434115    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-tmp-dir\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434132    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1226feab-5f42-4977-af53-d9d2957bd99e-host-var-lib-kubelet\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434140    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59hgb\" (UniqueName: \"kubernetes.io/projected/2db232de-e5b6-4cb6-99e0-32fa465d7003-kube-api-access-59hgb\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.436437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-systemd\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434188    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-env-overrides\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434213    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-serviceca\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434393    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/622ef6b4-0643-4162-b807-27de4bcd3ffd-run-systemd\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434421    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-tmp-dir\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434666    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-serviceca\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.434750    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/622ef6b4-0643-4162-b807-27de4bcd3ffd-env-overrides\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.435691    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97d954b6-fe98-486b-a51e-a3020d766739-agent-certs\") pod \"konnectivity-agent-fjq4d\" (UID: \"97d954b6-fe98-486b-a51e-a3020d766739\") " pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.435689    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2db232de-e5b6-4cb6-99e0-32fa465d7003-etc-tuned\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.435967    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/622ef6b4-0643-4162-b807-27de4bcd3ffd-ovn-node-metrics-cert\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.437109 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.436525    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db232de-e5b6-4cb6-99e0-32fa465d7003-tmp\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.441021 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.441005    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:16:47.441102 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.441025    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:16:47.441102 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.441083    2556 projected.go:196] Error preparing data for projected volume kube-api-access-n28z8 for pod openshift-network-diagnostics/network-check-target-52pn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:47.441200 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:47.441150    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8 podName:50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f nodeName:}" failed. No retries permitted until 2026-07-16 22:16:47.941132045 +0000 UTC m=+2.090606992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n28z8" (UniqueName: "kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8") pod "network-check-target-52pn4" (UID: "50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:47.443005 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.442986    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjq24\" (UniqueName: \"kubernetes.io/projected/fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3-kube-api-access-fjq24\") pod \"node-resolver-7nft5\" (UID: \"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3\") " pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.443371 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.443342    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8ld8\" (UniqueName: \"kubernetes.io/projected/622ef6b4-0643-4162-b807-27de4bcd3ffd-kube-api-access-n8ld8\") pod \"ovnkube-node-8k4z4\" (UID: \"622ef6b4-0643-4162-b807-27de4bcd3ffd\") " pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.443480 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.443461    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlbb4\" (UniqueName: \"kubernetes.io/projected/1226feab-5f42-4977-af53-d9d2957bd99e-kube-api-access-qlbb4\") pod \"multus-dbspd\" (UID: \"1226feab-5f42-4977-af53-d9d2957bd99e\") " pod="openshift-multus/multus-dbspd"
Jul 16 22:16:47.444301 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.444283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59hgb\" (UniqueName: \"kubernetes.io/projected/2db232de-e5b6-4cb6-99e0-32fa465d7003-kube-api-access-59hgb\") pod \"tuned-85pdd\" (UID: \"2db232de-e5b6-4cb6-99e0-32fa465d7003\") " pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.444384 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.444310    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxdwk\" (UniqueName: \"kubernetes.io/projected/4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7-kube-api-access-cxdwk\") pod \"node-ca-8gvtz\" (UID: \"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7\") " pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:47.547917 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.547818    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg"
Jul 16 22:16:47.556380 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.556355    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8c3fb4_78f2_4f65_a43c_86afbe4cc136.slice/crio-7a428f474d50c1f62520e1f496ccbd751893ed5467c7fcd2f4e698eee3245ce9 WatchSource:0}: Error finding container 7a428f474d50c1f62520e1f496ccbd751893ed5467c7fcd2f4e698eee3245ce9: Status 404 returned error can't find the container with id 7a428f474d50c1f62520e1f496ccbd751893ed5467c7fcd2f4e698eee3245ce9
Jul 16 22:16:47.559493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.559474    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cptpd"
Jul 16 22:16:47.566450 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.566425    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f94f52e_7e1a_4d9b_ad52_84bc9e5bd584.slice/crio-7731dfa4fc06dfe2f4576b94ffde37023f3f0937db9fc25a2cd8dc5a72fd2584 WatchSource:0}: Error finding container 7731dfa4fc06dfe2f4576b94ffde37023f3f0937db9fc25a2cd8dc5a72fd2584: Status 404 returned error can't find the container with id 7731dfa4fc06dfe2f4576b94ffde37023f3f0937db9fc25a2cd8dc5a72fd2584
Jul 16 22:16:47.567416 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.567399    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-28mf2"
Jul 16 22:16:47.574927 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.574885    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ee481c_767b_4a1c_9dd3_57a1e8f56d2b.slice/crio-183689f765eeabbed5577ce86d7e39ad83853ae5325e7a09309bcc5ebb3ad526 WatchSource:0}: Error finding container 183689f765eeabbed5577ce86d7e39ad83853ae5325e7a09309bcc5ebb3ad526: Status 404 returned error can't find the container with id 183689f765eeabbed5577ce86d7e39ad83853ae5325e7a09309bcc5ebb3ad526
Jul 16 22:16:47.589554 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.589525    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:16:47.596746 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.596713    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d954b6_fe98_486b_a51e_a3020d766739.slice/crio-5714147447d4650db77fb9319e07738743843e702213f9f56c648894a14bc6a0 WatchSource:0}: Error finding container 5714147447d4650db77fb9319e07738743843e702213f9f56c648894a14bc6a0: Status 404 returned error can't find the container with id 5714147447d4650db77fb9319e07738743843e702213f9f56c648894a14bc6a0
Jul 16 22:16:47.608999 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.608981    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-85pdd"
Jul 16 22:16:47.610165 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.610148    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:16:47.615950 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.615925    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db232de_e5b6_4cb6_99e0_32fa465d7003.slice/crio-d2a15039f45936462defc06672481855cc9733a778c5fa9e873391fa3d78d44d WatchSource:0}: Error finding container d2a15039f45936462defc06672481855cc9733a778c5fa9e873391fa3d78d44d: Status 404 returned error can't find the container with id d2a15039f45936462defc06672481855cc9733a778c5fa9e873391fa3d78d44d
Jul 16 22:16:47.625048 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.625031    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7nft5"
Jul 16 22:16:47.632819 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.632796    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc6c432_8858_4d49_b6ba_5c44ac2ed8d3.slice/crio-4f43603adf8defbf6897bb7a0b7451ac5c972910c6b68e5bb83085141851d2aa WatchSource:0}: Error finding container 4f43603adf8defbf6897bb7a0b7451ac5c972910c6b68e5bb83085141851d2aa: Status 404 returned error can't find the container with id 4f43603adf8defbf6897bb7a0b7451ac5c972910c6b68e5bb83085141851d2aa
Jul 16 22:16:47.647278 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.647254    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:16:47.654428 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:47.654405    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622ef6b4_0643_4162_b807_27de4bcd3ffd.slice/crio-5ec4104138924f85fd287914f0b6ffd10f86cb46833ee6113e8d39df3b00cfd2 WatchSource:0}: Error finding container 5ec4104138924f85fd287914f0b6ffd10f86cb46833ee6113e8d39df3b00cfd2: Status 404 returned error can't find the container with id 5ec4104138924f85fd287914f0b6ffd10f86cb46833ee6113e8d39df3b00cfd2
Jul 16 22:16:47.660075 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:47.660053    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8gvtz"
Jul 16 22:16:49.142021 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.141853    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dbspd"
Jul 16 22:16:49.144780 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:49.144755    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4ac9ef_39d7_4fe0_b11a_2346d6e6a7e7.slice/crio-bc9843cc3c27d06e270f363800cfa75fee88bd5b88b90e9a8c35e51c134a624e WatchSource:0}: Error finding container bc9843cc3c27d06e270f363800cfa75fee88bd5b88b90e9a8c35e51c134a624e: Status 404 returned error can't find the container with id bc9843cc3c27d06e270f363800cfa75fee88bd5b88b90e9a8c35e51c134a624e
Jul 16 22:16:49.149060 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:16:49.149039    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1226feab_5f42_4977_af53_d9d2957bd99e.slice/crio-32cf2fc27d7f38a308822d445b80ecde67b9e9862ae4afb60ad3d849cf39d256 WatchSource:0}: Error finding container 32cf2fc27d7f38a308822d445b80ecde67b9e9862ae4afb60ad3d849cf39d256: Status 404 returned error can't find the container with id 32cf2fc27d7f38a308822d445b80ecde67b9e9862ae4afb60ad3d849cf39d256
Jul 16 22:16:49.313487 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.313404    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:49.313623 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.313521    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:49.313623 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.313570    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs podName:b45c42e2-acce-454f-8aa5-59e0b573e914 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:50.313556025 +0000 UTC m=+2.986815013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs") pod "network-metrics-daemon-fd92h" (UID: "b45c42e2-acce-454f-8aa5-59e0b573e914") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:49.414410 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.414382    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:49.414664 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.414494    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:49.414664 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.414558    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret podName:c4d9b940-f340-4329-aea0-6252d6d331d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:50.414538984 +0000 UTC m=+3.087797982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret") pod "global-pull-secret-syncer-hhz6k" (UID: "c4d9b940-f340-4329-aea0-6252d6d331d0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:49.515199 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.515165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:49.515334 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.515291    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:16:49.515334 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.515307    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:16:49.515334 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.515316    2556 projected.go:196] Error preparing data for projected volume kube-api-access-n28z8 for pod openshift-network-diagnostics/network-check-target-52pn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:49.515457 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.515363    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8 podName:50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f nodeName:}" failed. No retries permitted until 2026-07-16 22:16:50.515350241 +0000 UTC m=+3.188609227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n28z8" (UniqueName: "kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8") pod "network-check-target-52pn4" (UID: "50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:49.746009 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.745741    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 22:11:48 +0000 UTC" deadline="2028-02-24 05:30:39.345650156 +0000 UTC"
Jul 16 22:16:49.746009 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.745784    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14095h13m49.599871365s"
Jul 16 22:16:49.806471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.806441    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:49.806648 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:49.806582    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:16:49.832446 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.832242    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8gvtz" event={"ID":"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7","Type":"ContainerStarted","Data":"bc9843cc3c27d06e270f363800cfa75fee88bd5b88b90e9a8c35e51c134a624e"}
Jul 16 22:16:49.834386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.834364    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:16:49.838617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.838582    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"5ec4104138924f85fd287914f0b6ffd10f86cb46833ee6113e8d39df3b00cfd2"}
Jul 16 22:16:49.845756 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.845725    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7nft5" event={"ID":"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3","Type":"ContainerStarted","Data":"4f43603adf8defbf6897bb7a0b7451ac5c972910c6b68e5bb83085141851d2aa"}
Jul 16 22:16:49.861725 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.861585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-85pdd" event={"ID":"2db232de-e5b6-4cb6-99e0-32fa465d7003","Type":"ContainerStarted","Data":"d2a15039f45936462defc06672481855cc9733a778c5fa9e873391fa3d78d44d"}
Jul 16 22:16:49.866485 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.866451    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjq4d" event={"ID":"97d954b6-fe98-486b-a51e-a3020d766739","Type":"ContainerStarted","Data":"5714147447d4650db77fb9319e07738743843e702213f9f56c648894a14bc6a0"}
Jul 16 22:16:49.872877 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.872849    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-28mf2" event={"ID":"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b","Type":"ContainerStarted","Data":"183689f765eeabbed5577ce86d7e39ad83853ae5325e7a09309bcc5ebb3ad526"}
Jul 16 22:16:49.881572 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.881544    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal" event={"ID":"a88942796a5b6b2c5f36a7e709e10120","Type":"ContainerStarted","Data":"56df0a1fcef21f95c4b785bf46e48536ae0ca69a53498c5d6f644c0c98277151"}
Jul 16 22:16:49.885340 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.885310    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal" event={"ID":"6a91334f2b2daa9817292f0040684ba7","Type":"ContainerStarted","Data":"c11159515a4706f8986239a9e2ea2a8b5a505dffab2388c348c4cb019ce07b5f"}
Jul 16 22:16:49.890986 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.890963    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dbspd" event={"ID":"1226feab-5f42-4977-af53-d9d2957bd99e","Type":"ContainerStarted","Data":"32cf2fc27d7f38a308822d445b80ecde67b9e9862ae4afb60ad3d849cf39d256"}
Jul 16 22:16:49.898367 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.898288    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerStarted","Data":"7731dfa4fc06dfe2f4576b94ffde37023f3f0937db9fc25a2cd8dc5a72fd2584"}
Jul 16 22:16:49.908756 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:49.908729    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg" event={"ID":"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136","Type":"ContainerStarted","Data":"7a428f474d50c1f62520e1f496ccbd751893ed5467c7fcd2f4e698eee3245ce9"}
Jul 16 22:16:50.320186 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.320135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:50.320397 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.320311    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:50.320464 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.320398    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs podName:b45c42e2-acce-454f-8aa5-59e0b573e914 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:52.320375368 +0000 UTC m=+4.993634354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs") pod "network-metrics-daemon-fd92h" (UID: "b45c42e2-acce-454f-8aa5-59e0b573e914") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:50.421296 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.421248    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:50.421481 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.421449    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:50.421542 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.421512    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret podName:c4d9b940-f340-4329-aea0-6252d6d331d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:52.421492867 +0000 UTC m=+5.094751867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret") pod "global-pull-secret-syncer-hhz6k" (UID: "c4d9b940-f340-4329-aea0-6252d6d331d0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:50.522293 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.522256    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:50.522467 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.522415    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:16:50.522467 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.522434    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:16:50.522467 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.522447    2556 projected.go:196] Error preparing data for projected volume kube-api-access-n28z8 for pod openshift-network-diagnostics/network-check-target-52pn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:50.522632 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.522511    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8 podName:50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f nodeName:}" failed. No retries permitted until 2026-07-16 22:16:52.522493386 +0000 UTC m=+5.195752378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n28z8" (UniqueName: "kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8") pod "network-check-target-52pn4" (UID: "50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:50.747002 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.746891    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 22:11:48 +0000 UTC" deadline="2028-01-17 15:34:41.048056233 +0000 UTC"
Jul 16 22:16:50.747002 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.746956    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13193h17m50.301103883s"
Jul 16 22:16:50.806043 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.806013    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:50.806217 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.806027    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:50.806217 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.806145    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:16:50.806345 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:50.806224    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:16:50.902358 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:50.901969    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:16:51.762446 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:51.762238    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:16:51.806770 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:51.806260    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:51.806770 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:51.806411    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:16:52.340697 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:52.340656    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:52.340903 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.340860    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:52.340976 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.340923    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs podName:b45c42e2-acce-454f-8aa5-59e0b573e914 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:56.340905309 +0000 UTC m=+9.014164300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs") pod "network-metrics-daemon-fd92h" (UID: "b45c42e2-acce-454f-8aa5-59e0b573e914") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:52.442434 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:52.441837    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:52.442434 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.442021    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:52.442434 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.442086    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret podName:c4d9b940-f340-4329-aea0-6252d6d331d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:56.442065997 +0000 UTC m=+9.115324987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret") pod "global-pull-secret-syncer-hhz6k" (UID: "c4d9b940-f340-4329-aea0-6252d6d331d0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:52.542577 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:52.542540    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:52.542768 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.542745    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:16:52.542854 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.542769    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:16:52.542854 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.542783    2556 projected.go:196] Error preparing data for projected volume kube-api-access-n28z8 for pod openshift-network-diagnostics/network-check-target-52pn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:52.542854 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.542844    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8 podName:50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f nodeName:}" failed. No retries permitted until 2026-07-16 22:16:56.542826767 +0000 UTC m=+9.216085756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n28z8" (UniqueName: "kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8") pod "network-check-target-52pn4" (UID: "50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:52.806721 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:52.806022    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:52.806721 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.806171    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:16:52.806721 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:52.806546    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:52.806721 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:52.806633    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:16:53.805883 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:53.805854    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:53.806037 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:53.805988    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:16:54.805964 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:54.805932    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:54.806389 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:54.806043    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:16:54.806389 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:54.806211    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:54.806389 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:54.806347    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:16:55.809390 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:55.809358    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:55.809830 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:55.809486    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:16:56.377347 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:56.377295    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:56.377545 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.377495    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:56.377604 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.377564    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs podName:b45c42e2-acce-454f-8aa5-59e0b573e914 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:04.377544983 +0000 UTC m=+17.050803992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs") pod "network-metrics-daemon-fd92h" (UID: "b45c42e2-acce-454f-8aa5-59e0b573e914") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:16:56.478540 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:56.478500    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:56.478721 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.478643    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:56.478721 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.478693    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret podName:c4d9b940-f340-4329-aea0-6252d6d331d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:04.478679166 +0000 UTC m=+17.151938152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret") pod "global-pull-secret-syncer-hhz6k" (UID: "c4d9b940-f340-4329-aea0-6252d6d331d0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:16:56.579327 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:56.579285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:56.579523 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.579503    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:16:56.579578 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.579531    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:16:56.579578 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.579547    2556 projected.go:196] Error preparing data for projected volume kube-api-access-n28z8 for pod openshift-network-diagnostics/network-check-target-52pn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:56.579652 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.579610    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8 podName:50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:04.579589472 +0000 UTC m=+17.252848462 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n28z8" (UniqueName: "kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8") pod "network-check-target-52pn4" (UID: "50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:16:56.805743 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:56.805663    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:56.805896 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.805793    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:16:56.805896 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:56.805846    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:56.806015 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:56.805956    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:16:57.806987 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:57.806515    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:57.806987 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:57.806647    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:16:58.806155 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:58.806101    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:16:58.806155 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:58.806139    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:16:58.806404 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:58.806241    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:16:58.806404 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:58.806365    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:16:59.806054 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:16:59.805817    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:16:59.806508 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:16:59.806175    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:00.806202 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:00.806157    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:00.806202 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:00.806193    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:00.806687 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:00.806279    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:00.806687 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:00.806432    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:01.806280 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:01.806244    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:01.806731 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:01.806374    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:02.805844 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:02.805807    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:02.806022 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:02.805807    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:02.806022 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:02.805943    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:02.806022 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:02.806006    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:03.805480 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:03.805437    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:03.805948 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:03.805601    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:04.441351 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:04.441295    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:04.441523 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.441448    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:04.441523 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.441509    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs podName:b45c42e2-acce-454f-8aa5-59e0b573e914 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:20.441492411 +0000 UTC m=+33.114751398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs") pod "network-metrics-daemon-fd92h" (UID: "b45c42e2-acce-454f-8aa5-59e0b573e914") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:04.542558 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:04.542529    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:04.542701 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.542654    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:04.542754 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.542723    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret podName:c4d9b940-f340-4329-aea0-6252d6d331d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:20.542704353 +0000 UTC m=+33.215963342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret") pod "global-pull-secret-syncer-hhz6k" (UID: "c4d9b940-f340-4329-aea0-6252d6d331d0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:04.642875 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:04.642839    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:04.643063 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.643008    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:04.643063 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.643028    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:04.643063 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.643037    2556 projected.go:196] Error preparing data for projected volume kube-api-access-n28z8 for pod openshift-network-diagnostics/network-check-target-52pn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:04.643226 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.643098    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8 podName:50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:20.643080287 +0000 UTC m=+33.316339277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n28z8" (UniqueName: "kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8") pod "network-check-target-52pn4" (UID: "50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:04.805720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:04.805688    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:04.806193 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.805818    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:04.806193 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:04.805873    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:04.806193 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:04.805962    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:05.808740 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:05.808711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:05.809183 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:05.808834    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:06.806083 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:06.806046    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:06.806083 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:06.806090    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:06.806334 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:06.806195    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:06.806334 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:06.806305    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:07.807053 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:07.807012    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:07.807463 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:07.807161    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:08.806079 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:08.806040    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:08.806079 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:08.806065    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:08.806318 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:08.806185    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:08.806318 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:08.806302    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:09.805652 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:09.805631    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:09.806013 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:09.805733    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:09.976667 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:09.976635    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal" event={"ID":"a88942796a5b6b2c5f36a7e709e10120","Type":"ContainerStarted","Data":"23997f461ec5b6f936011e606f92109551737453bcd130605070ae4af8118a25"}
Jul 16 22:17:09.980193 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:09.980162    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dbspd" event={"ID":"1226feab-5f42-4977-af53-d9d2957bd99e","Type":"ContainerStarted","Data":"4e2cdef43ebe413c5a46107fcf6345e3e50d83b63ea99d5dd48ad0ef22ba582b"}
Jul 16 22:17:09.984554 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:09.984369    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"c09edf8db574baf031631829cbff8bc6691c02c5610536e1a24e1c1f4e8dc602"}
Jul 16 22:17:09.984554 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:09.984437    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"0b98a2d63c23b774cc559fa7374a182ce2b8ad4b781f179ce479178bef11f2c8"}
Jul 16 22:17:09.985668 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:09.985648    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-85pdd" event={"ID":"2db232de-e5b6-4cb6-99e0-32fa465d7003","Type":"ContainerStarted","Data":"5adb766dd7e17ccf3acedef9df3a05a46f85991a4df0f7f4cde84986f5bc21a9"}
Jul 16 22:17:09.991913 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:09.991868    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-105.ec2.internal" podStartSLOduration=21.991856587 podStartE2EDuration="21.991856587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:09.991690565 +0000 UTC m=+22.664949595" watchObservedRunningTime="2026-07-16 22:17:09.991856587 +0000 UTC m=+22.665115594"
Jul 16 22:17:10.008962 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.008919    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dbspd" podStartSLOduration=1.533176853 podStartE2EDuration="22.008906552s" totalImagesPullingTime="20.475729699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:10.008093738 +0000 UTC m=+22.681352777" watchObservedRunningTime="2026-07-16 22:17:10.008906552 +0000 UTC m=+22.682165560"
Jul 16 22:17:10.024384 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.024316    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-85pdd" podStartSLOduration=1.5408338160000001 podStartE2EDuration="22.02430108s" totalImagesPullingTime="20.483467264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:10.02402267 +0000 UTC m=+22.697281702" watchObservedRunningTime="2026-07-16 22:17:10.02430108 +0000 UTC m=+22.697560088"
Jul 16 22:17:10.806335 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.806143    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:10.807045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.806189    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:10.807045 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:10.806411    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:10.807045 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:10.806511    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:10.989466 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.989427    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8gvtz" event={"ID":"4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7","Type":"ContainerStarted","Data":"3ed763e5b6798dfd29cd2a26591fa1cf27a898053f31586c3c8d4054c8edc746"}
Jul 16 22:17:10.991982 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.991962    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:17:10.992306 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.992285    2556 generic.go:356] "Generic (PLEG): container finished" podID="622ef6b4-0643-4162-b807-27de4bcd3ffd" containerID="c09edf8db574baf031631829cbff8bc6691c02c5610536e1a24e1c1f4e8dc602" exitCode=1
Jul 16 22:17:10.992395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.992360    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerDied","Data":"c09edf8db574baf031631829cbff8bc6691c02c5610536e1a24e1c1f4e8dc602"}
Jul 16 22:17:10.992442 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.992421    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"e513c0275b4c0418d7dba79b338a2667d0360fe65dae88a567b2797c4ea9c724"}
Jul 16 22:17:10.992493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.992451    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"b7247aea1871a3fa139219a745def144924a2ba62b4ffe08c3e3bf36a98bea05"}
Jul 16 22:17:10.992493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.992486    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"18d6e4af18fc41ec82a91957ade002d1dc855809200125069e4f6f79bb4ad897"}
Jul 16 22:17:10.992568 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.992516    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"188464d8a019ecd1f5b1354086488d8677360d9a713c3dae506248cf326f499f"}
Jul 16 22:17:10.993716 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.993696    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7nft5" event={"ID":"fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3","Type":"ContainerStarted","Data":"c7fd27b0b946e7fd1c19924c40853c653eefd715c1e118de2c2b0e7198e8bf7f"}
Jul 16 22:17:10.995094 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.995074    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjq4d" event={"ID":"97d954b6-fe98-486b-a51e-a3020d766739","Type":"ContainerStarted","Data":"77fd09a0f54b27f19b8d7d868ac1ec35504925057df3c4e3ea00454623c357d8"}
Jul 16 22:17:10.996432 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.996411    2556 generic.go:356] "Generic (PLEG): container finished" podID="6a91334f2b2daa9817292f0040684ba7" containerID="d7a0f938871ab4d6ca55fd3935c98cef409b65b838126a2358869ff21a641a9c" exitCode=0
Jul 16 22:17:10.996510 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.996434    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal" event={"ID":"6a91334f2b2daa9817292f0040684ba7","Type":"ContainerDied","Data":"d7a0f938871ab4d6ca55fd3935c98cef409b65b838126a2358869ff21a641a9c"}
Jul 16 22:17:10.997913 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.997895    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584" containerID="a5ba3be3e0c1df0fb03443ae62fb7eff23e0e851ce30ae439474c8c13c910492" exitCode=0
Jul 16 22:17:10.998008 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.997955    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerDied","Data":"a5ba3be3e0c1df0fb03443ae62fb7eff23e0e851ce30ae439474c8c13c910492"}
Jul 16 22:17:10.999346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:10.999328    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg" event={"ID":"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136","Type":"ContainerStarted","Data":"eaf7400e2766898b0bac50c8d4b9fbd45c40610edf2f6617ec00a7ea1b0e622f"}
Jul 16 22:17:11.005082 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:11.005045    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8gvtz" podStartSLOduration=2.595367149 podStartE2EDuration="23.005032895s" totalImagesPullingTime="20.409665746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:11.004574464 +0000 UTC m=+23.677833472" watchObservedRunningTime="2026-07-16 22:17:11.005032895 +0000 UTC m=+23.678291903"
Jul 16 22:17:11.020351 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:11.020289    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fjq4d" podStartSLOduration=10.646597559 podStartE2EDuration="23.020274915s" totalImagesPullingTime="12.373677356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:11.019766136 +0000 UTC m=+23.693025145" watchObservedRunningTime="2026-07-16 22:17:11.020274915 +0000 UTC m=+23.693533920"
Jul 16 22:17:11.036880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:11.036836    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7nft5" podStartSLOduration=2.591281962 podStartE2EDuration="23.036818702s" totalImagesPullingTime="20.44553674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:11.036615595 +0000 UTC m=+23.709874613" watchObservedRunningTime="2026-07-16 22:17:11.036818702 +0000 UTC m=+23.710077711"
Jul 16 22:17:11.806195 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:11.806155    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:11.806436 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:11.806315    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:12.002845 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.002803    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-28mf2" event={"ID":"24ee481c-767b-4a1c-9dd3-57a1e8f56d2b","Type":"ContainerStarted","Data":"286bae2387bef001064add10360050303e6ffb25b302d2dfe15c97824b1464dc"}
Jul 16 22:17:12.004809 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.004776    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal" event={"ID":"6a91334f2b2daa9817292f0040684ba7","Type":"ContainerStarted","Data":"807669f0bda49993f558a24c3dcc718ad83752a6827ed14450b7ea552cc1bf44"}
Jul 16 22:17:12.018251 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.018201    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-28mf2" podStartSLOduration=4.499758574 podStartE2EDuration="25.018185328s" totalImagesPullingTime="20.518426754s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:12.017734191 +0000 UTC m=+24.690993205" watchObservedRunningTime="2026-07-16 22:17:12.018185328 +0000 UTC m=+24.691444335"
Jul 16 22:17:12.032492 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.032428    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-105.ec2.internal" podStartSLOduration=24.032406049 podStartE2EDuration="24.032406049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:12.03219661 +0000 UTC m=+24.705455646" watchObservedRunningTime="2026-07-16 22:17:12.032406049 +0000 UTC m=+24.705665060"
Jul 16 22:17:12.041167 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.041139    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 22:17:12.643621 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.643531    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:17:12.644198 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.644182    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:17:12.770862 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.770773    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T22:17:12.041163463Z","UUID":"85ff7b66-70e4-47c2-97c5-09c62d823bb9","Handler":null,"Name":"","Endpoint":""}
Jul 16 22:17:12.772447 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.772422    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 22:17:12.772563 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.772461    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 22:17:12.806224 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.806194    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:12.806396 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:12.806203    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:12.806396 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:12.806297    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:12.806396 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:12.806375    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:13.010734 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:13.010657    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:17:13.011167 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:13.011091    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"7a7673fa64cca08a01f3b9e372a29633383f422ac85ec0c934d90ffd5c666990"}
Jul 16 22:17:13.013090 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:13.013010    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg" event={"ID":"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136","Type":"ContainerStarted","Data":"08f0318539895456789e6aed6d9f1f31b77eb2f81da7a892a34037f6e4935335"}
Jul 16 22:17:13.013650 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:13.013463    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:17:13.013968 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:13.013953    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fjq4d"
Jul 16 22:17:13.805537 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:13.805505    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:13.806165 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:13.805988    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:14.806097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:14.806061    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:14.806097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:14.806084    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:14.806735 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:14.806201    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:14.806735 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:14.806312    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:15.021129 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.021020    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:17:15.021512 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.021483    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"e52413a73f9a10acd8fa99b4cd371129f8a50bbdffd2559fc27464782b4579dc"}
Jul 16 22:17:15.021929 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.021902    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:17:15.022078 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.022060    2556 scope.go:117] "RemoveContainer" containerID="c09edf8db574baf031631829cbff8bc6691c02c5610536e1a24e1c1f4e8dc602"
Jul 16 22:17:15.023814 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.023794    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg" event={"ID":"ad8c3fb4-78f2-4f65-a43c-86afbe4cc136","Type":"ContainerStarted","Data":"456c6a1b360d017a90d2b5d298d3cad599961a1e294a63a32cc669efd95671a9"}
Jul 16 22:17:15.040346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.040317    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:17:15.067796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.067756    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4qlhg" podStartSLOduration=3.895172708 podStartE2EDuration="28.067741019s" totalImagesPullingTime="24.172568311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:15.067528595 +0000 UTC m=+27.740787604" watchObservedRunningTime="2026-07-16 22:17:15.067741019 +0000 UTC m=+27.741000026"
Jul 16 22:17:15.805928 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:15.805892    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:15.806171 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:15.806013    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:16.030628 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.030605    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:17:16.031096 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.031060    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" event={"ID":"622ef6b4-0643-4162-b807-27de4bcd3ffd","Type":"ContainerStarted","Data":"39cabbb7438fc3f0f3d4ea683de9dcb89a9396ff6fd199e36bf31671bd9198f2"}
Jul 16 22:17:16.031306 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.031240    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:17:16.031404 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.031384    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:17:16.048416 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.048388    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:17:16.062053 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.061952    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4" podStartSLOduration=7.569771678 podStartE2EDuration="28.061937303s" totalImagesPullingTime="20.492165625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:16.061276951 +0000 UTC m=+28.734535971" watchObservedRunningTime="2026-07-16 22:17:16.061937303 +0000 UTC m=+28.735196311"
Jul 16 22:17:16.796689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.796471    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fd92h"]
Jul 16 22:17:16.796898 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.796787    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:16.796961 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:16.796885    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:16.800311 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.800266    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhz6k"]
Jul 16 22:17:16.800471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.800337    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52pn4"]
Jul 16 22:17:16.800471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.800418    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:16.800580 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:16.800507    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:16.800580 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:16.800567    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:16.800684 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:16.800626    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:18.037553 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:18.037519    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584" containerID="e530c786c96606e3e6346db61a6a5777f7ac55eb2eb03f26be63f9a3ab19be85" exitCode=0
Jul 16 22:17:18.037963 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:18.037597    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerDied","Data":"e530c786c96606e3e6346db61a6a5777f7ac55eb2eb03f26be63f9a3ab19be85"}
Jul 16 22:17:18.805893 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:18.805860    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:18.805893 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:18.805887    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:18.806133 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:18.805890    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:18.806133 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:18.805970    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:18.806133 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:18.806064    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:18.806255 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:18.806157    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:20.043591 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.043557    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584" containerID="af3f297995f33f215a4ac706bb7a0b1dc4ef7a9ea6f098b75aa1adad54e5ed64" exitCode=0
Jul 16 22:17:20.043933 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.043606    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerDied","Data":"af3f297995f33f215a4ac706bb7a0b1dc4ef7a9ea6f098b75aa1adad54e5ed64"}
Jul 16 22:17:20.466274 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.466174    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:20.466446 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.466337    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:20.466446 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.466419    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs podName:b45c42e2-acce-454f-8aa5-59e0b573e914 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:52.466400541 +0000 UTC m=+65.139659528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs") pod "network-metrics-daemon-fd92h" (UID: "b45c42e2-acce-454f-8aa5-59e0b573e914") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:17:20.566736 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.566693    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:20.566889 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.566841    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:20.566952 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.566906    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret podName:c4d9b940-f340-4329-aea0-6252d6d331d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:52.566889893 +0000 UTC m=+65.240148880 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret") pod "global-pull-secret-syncer-hhz6k" (UID: "c4d9b940-f340-4329-aea0-6252d6d331d0") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:17:20.667305 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.667270    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:20.667477 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.667459    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:17:20.667519 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.667485    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:17:20.667519 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.667495    2556 projected.go:196] Error preparing data for projected volume kube-api-access-n28z8 for pod openshift-network-diagnostics/network-check-target-52pn4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:20.667581 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.667550    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8 podName:50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:52.667531111 +0000 UTC m=+65.340790098 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n28z8" (UniqueName: "kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8") pod "network-check-target-52pn4" (UID: "50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:17:20.805567 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.805537    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:20.805745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.805682    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:20.805745 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.805700    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-52pn4" podUID="50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f"
Jul 16 22:17:20.805745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:20.805715    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:20.805911 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.805798    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hhz6k" podUID="c4d9b940-f340-4329-aea0-6252d6d331d0"
Jul 16 22:17:20.805911 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:20.805898    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fd92h" podUID="b45c42e2-acce-454f-8aa5-59e0b573e914"
Jul 16 22:17:21.048316 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:21.048284    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584" containerID="0edab5c016321501f1b34f73de1057b967e546b06a9fcdfeb46a98cc608fb803" exitCode=0
Jul 16 22:17:21.048689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:21.048335    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerDied","Data":"0edab5c016321501f1b34f73de1057b967e546b06a9fcdfeb46a98cc608fb803"}
Jul 16 22:17:22.635026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.634996    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-105.ec2.internal" event="NodeReady"
Jul 16 22:17:22.635560 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.635165    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 22:17:22.674535 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.674412    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-779c67dc6c-q59db"]
Jul 16 22:17:22.676616 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.676592    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.678988 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.678969    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 22:17:22.679674 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.679475    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 22:17:22.680081 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.680063    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 22:17:22.680190 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.680101    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cmptb\""
Jul 16 22:17:22.687885 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.687850    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 22:17:22.689283 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.689262    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2zxmt"]
Jul 16 22:17:22.691266 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.691251    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-49m5t"]
Jul 16 22:17:22.691375 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.691361    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.692846 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.692830    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:22.693688 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.693670    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 22:17:22.693927 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.693676    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4hsw2\""
Jul 16 22:17:22.694071 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.694051    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 22:17:22.694792 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.694771    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779c67dc6c-q59db"]
Jul 16 22:17:22.695559 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.695537    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ssz9k\""
Jul 16 22:17:22.695645 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.695605    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 22:17:22.695856 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.695842    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 22:17:22.696186 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.696171    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 22:17:22.701866 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.701844    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-49m5t"]
Jul 16 22:17:22.708652 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.708633    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zxmt"]
Jul 16 22:17:22.784605 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784571    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13f7c116-08cb-4fe3-9c54-ab12fb15b424-cert\") pod \"ingress-canary-49m5t\" (UID: \"13f7c116-08cb-4fe3-9c54-ab12fb15b424\") " pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:22.784786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784614    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.784786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784670    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f81de92-62c5-4b55-833d-1d1653130da3-ca-trust-extracted\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.784786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784697    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-trusted-ca\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.784786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784728    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.784786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784742    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-installation-pull-secrets\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.784786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784760    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-registry-certificates\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.784786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-tmp-dir\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.785044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784803    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-config-volume\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.785044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784834    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-bound-sa-token\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.785044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784856    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m28kz\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-kube-api-access-m28kz\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.785044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-image-registry-private-configuration\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.785044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjdv\" (UniqueName: \"kubernetes.io/projected/13f7c116-08cb-4fe3-9c54-ab12fb15b424-kube-api-access-hdjdv\") pod \"ingress-canary-49m5t\" (UID: \"13f7c116-08cb-4fe3-9c54-ab12fb15b424\") " pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:22.785044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.784967    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7lzr\" (UniqueName: \"kubernetes.io/projected/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-kube-api-access-v7lzr\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.805676 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.805643    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:22.805847 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.805642    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:22.805847 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.805642    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:22.809026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.808455    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 22:17:22.809026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.808521    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 22:17:22.809026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.808550    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 22:17:22.809026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.808580    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjst2\""
Jul 16 22:17:22.809026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.808596    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 22:17:22.809026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.808892    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqtwh\""
Jul 16 22:17:22.886069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886033    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.886069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886071    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-installation-pull-secrets\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.886328 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886102    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-registry-certificates\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.886328 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886143    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-tmp-dir\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.886328 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886168    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-config-volume\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.886328 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:22.886198    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 22:17:22.886328 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:22.886220    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779c67dc6c-q59db: secret "image-registry-tls" not found
Jul 16 22:17:22.886328 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:22.886281    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls podName:5f81de92-62c5-4b55-833d-1d1653130da3 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:23.386262516 +0000 UTC m=+36.059521506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls") pod "image-registry-779c67dc6c-q59db" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3") : secret "image-registry-tls" not found
Jul 16 22:17:22.886609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886200    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-bound-sa-token\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.886609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886445    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m28kz\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-kube-api-access-m28kz\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.886609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886477    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-image-registry-private-configuration\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.886609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886520    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjdv\" (UniqueName: \"kubernetes.io/projected/13f7c116-08cb-4fe3-9c54-ab12fb15b424-kube-api-access-hdjdv\") pod \"ingress-canary-49m5t\" (UID: \"13f7c116-08cb-4fe3-9c54-ab12fb15b424\") " pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:22.886609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886552    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7lzr\" (UniqueName: \"kubernetes.io/projected/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-kube-api-access-v7lzr\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.886609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886579    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13f7c116-08cb-4fe3-9c54-ab12fb15b424-cert\") pod \"ingress-canary-49m5t\" (UID: \"13f7c116-08cb-4fe3-9c54-ab12fb15b424\") " pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:22.886609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886598    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.886918 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f81de92-62c5-4b55-833d-1d1653130da3-ca-trust-extracted\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.886918 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.886656    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-trusted-ca\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.887154 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.887135    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-registry-certificates\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.887154 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.887147    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-tmp-dir\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.887284 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:22.887254    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:17:22.887335 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:22.887308    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls podName:a704ee01-ae37-4011-bb07-0a1fd7ca87b9 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:23.387290989 +0000 UTC m=+36.060549977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls") pod "dns-default-2zxmt" (UID: "a704ee01-ae37-4011-bb07-0a1fd7ca87b9") : secret "dns-default-metrics-tls" not found
Jul 16 22:17:22.887512 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.887488    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-config-volume\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.887582 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.887572    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-trusted-ca\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.887824 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.887641    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f81de92-62c5-4b55-833d-1d1653130da3-ca-trust-extracted\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.891013 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.890989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-image-registry-private-configuration\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.891013 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.891008    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-installation-pull-secrets\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.891389 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.891375    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13f7c116-08cb-4fe3-9c54-ab12fb15b424-cert\") pod \"ingress-canary-49m5t\" (UID: \"13f7c116-08cb-4fe3-9c54-ab12fb15b424\") " pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:22.895689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.895665    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-bound-sa-token\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:22.896243 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.896219    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7lzr\" (UniqueName: \"kubernetes.io/projected/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-kube-api-access-v7lzr\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:22.896243 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.896231    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjdv\" (UniqueName: \"kubernetes.io/projected/13f7c116-08cb-4fe3-9c54-ab12fb15b424-kube-api-access-hdjdv\") pod \"ingress-canary-49m5t\" (UID: \"13f7c116-08cb-4fe3-9c54-ab12fb15b424\") " pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:22.896395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:22.896229    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m28kz\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-kube-api-access-m28kz\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:23.007244 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:23.007161    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-49m5t"
Jul 16 22:17:23.166059 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:23.166021    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-49m5t"]
Jul 16 22:17:23.169585 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:23.169556    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f7c116_08cb_4fe3_9c54_ab12fb15b424.slice/crio-121b0aa0797ce121013c74ca7c6c97fe86771733ee7ac686bd997e9ae1ef87f3 WatchSource:0}: Error finding container 121b0aa0797ce121013c74ca7c6c97fe86771733ee7ac686bd997e9ae1ef87f3: Status 404 returned error can't find the container with id 121b0aa0797ce121013c74ca7c6c97fe86771733ee7ac686bd997e9ae1ef87f3
Jul 16 22:17:23.392065 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:23.392026    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:23.392252 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:23.392135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:23.392252 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:23.392212    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:17:23.392388 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:23.392253    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 22:17:23.392388 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:23.392268    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779c67dc6c-q59db: secret "image-registry-tls" not found
Jul 16 22:17:23.392388 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:23.392292    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls podName:a704ee01-ae37-4011-bb07-0a1fd7ca87b9 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:24.392271675 +0000 UTC m=+37.065530664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls") pod "dns-default-2zxmt" (UID: "a704ee01-ae37-4011-bb07-0a1fd7ca87b9") : secret "dns-default-metrics-tls" not found
Jul 16 22:17:23.392388 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:23.392328    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls podName:5f81de92-62c5-4b55-833d-1d1653130da3 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:24.392302806 +0000 UTC m=+37.065561796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls") pod "image-registry-779c67dc6c-q59db" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3") : secret "image-registry-tls" not found
Jul 16 22:17:24.057770 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:24.057739    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-49m5t" event={"ID":"13f7c116-08cb-4fe3-9c54-ab12fb15b424","Type":"ContainerStarted","Data":"121b0aa0797ce121013c74ca7c6c97fe86771733ee7ac686bd997e9ae1ef87f3"}
Jul 16 22:17:24.401897 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:24.401809    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:24.402075 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:24.401908    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:24.402075 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:24.401964    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:17:24.402075 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:24.402021    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls podName:a704ee01-ae37-4011-bb07-0a1fd7ca87b9 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:26.402007614 +0000 UTC m=+39.075266600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls") pod "dns-default-2zxmt" (UID: "a704ee01-ae37-4011-bb07-0a1fd7ca87b9") : secret "dns-default-metrics-tls" not found
Jul 16 22:17:24.402271 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:24.402068    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 22:17:24.402271 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:24.402092    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779c67dc6c-q59db: secret "image-registry-tls" not found
Jul 16 22:17:24.402271 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:24.402156    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls podName:5f81de92-62c5-4b55-833d-1d1653130da3 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:26.402139211 +0000 UTC m=+39.075398196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls") pod "image-registry-779c67dc6c-q59db" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3") : secret "image-registry-tls" not found
Jul 16 22:17:25.678165 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:25.678097    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:25.880780 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:25.880550    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:25.969830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:25.969742    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:26.022575 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:26.022530    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:26.418171 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:26.418135    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:26.418324 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:26.418197    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:26.418324 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:26.418293    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:17:26.418324 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:26.418298    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 22:17:26.418324 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:26.418326    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779c67dc6c-q59db: secret "image-registry-tls" not found
Jul 16 22:17:26.418525 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:26.418390    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls podName:a704ee01-ae37-4011-bb07-0a1fd7ca87b9 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:30.418367294 +0000 UTC m=+43.091626285 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls") pod "dns-default-2zxmt" (UID: "a704ee01-ae37-4011-bb07-0a1fd7ca87b9") : secret "dns-default-metrics-tls" not found
Jul 16 22:17:26.418525 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:26.418412    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls podName:5f81de92-62c5-4b55-833d-1d1653130da3 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:30.418402449 +0000 UTC m=+43.091661434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls") pod "image-registry-779c67dc6c-q59db" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3") : secret "image-registry-tls" not found
Jul 16 22:17:26.721058 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:26.720959    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:26.778956 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:26.778917    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:27.067325 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:27.067293    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-49m5t" event={"ID":"13f7c116-08cb-4fe3-9c54-ab12fb15b424","Type":"ContainerStarted","Data":"e2f789ee342abef978d0bc24d0391abd93656f824ff93017803adfd3bd1b2353"}
Jul 16 22:17:27.070075 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:27.070045    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584" containerID="19a1f64aa5dbe3eb4c6d88ceaeda6aa88cd543128099260e48a9ded1e3f5c420" exitCode=0
Jul 16 22:17:27.070238 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:27.070090    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerDied","Data":"19a1f64aa5dbe3eb4c6d88ceaeda6aa88cd543128099260e48a9ded1e3f5c420"}
Jul 16 22:17:27.084544 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:27.084492    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-49m5t" podStartSLOduration=1.417230383 podStartE2EDuration="5.084477038s" totalImagesPullingTime="3.667246655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:27.084255157 +0000 UTC m=+39.757514164" watchObservedRunningTime="2026-07-16 22:17:27.084477038 +0000 UTC m=+39.757736045"
Jul 16 22:17:28.075361 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.075324    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584" containerID="c99a50129441ad6a00b4aba94d79e3fa67e7a4fc704e35802f2bccd20d242d1a" exitCode=0
Jul 16 22:17:28.075791 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.075402    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerDied","Data":"c99a50129441ad6a00b4aba94d79e3fa67e7a4fc704e35802f2bccd20d242d1a"}
Jul 16 22:17:28.187843 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.187806    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"]
Jul 16 22:17:28.201429 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.201395    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"]
Jul 16 22:17:28.201575 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.201536    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"
Jul 16 22:17:28.204360 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.204332    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-lbjbl\""
Jul 16 22:17:28.217506 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.217479    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"]
Jul 16 22:17:28.217660 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.217611    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.220644 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.220619    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.220757 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.220670    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-f9tqc\""
Jul 16 22:17:28.221155 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.221140    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.221238 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.221219    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 16 22:17:28.221435 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.221343    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 16 22:17:28.235577 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.235551    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"]
Jul 16 22:17:28.235577 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.235582    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"]
Jul 16 22:17:28.235785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.235599    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"]
Jul 16 22:17:28.235785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.235616    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sf45x"]
Jul 16 22:17:28.235785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.235688    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.238303 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.238267    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:28.238436 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.238339    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.238436 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.238397    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 16 22:17:28.238579 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.238451    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7wd4s\""
Jul 16 22:17:28.238579 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.238501    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 16 22:17:28.238579 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.238540    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.257720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.257685    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sf45x"]
Jul 16 22:17:28.257855 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.257797    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.260182 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.260161    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nqjc5\""
Jul 16 22:17:28.260344 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.260190    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 16 22:17:28.260344 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.260199    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 16 22:17:28.260344 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.260216    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.260616 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.260603    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.266221 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.266201    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 16 22:17:28.290763 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.290734    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4b8xc"]
Jul 16 22:17:28.307910 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.307851    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"]
Jul 16 22:17:28.308087 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.308001    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.310638 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.310615    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.310783 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.310766    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 16 22:17:28.311013 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.310997    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.311097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.311001    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 16 22:17:28.311097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.311027    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rgslb\""
Jul 16 22:17:28.315859 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.315839    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 16 22:17:28.330037 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.329959    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4b8xc"]
Jul 16 22:17:28.330037 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.329993    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"]
Jul 16 22:17:28.330239 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.330080    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"
Jul 16 22:17:28.331175 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.331155    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twhww\" (UniqueName: \"kubernetes.io/projected/4a08215e-447d-4500-8136-3a196037bc48-kube-api-access-twhww\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.331228 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.331196    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a08215e-447d-4500-8136-3a196037bc48-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.331343 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.331319    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.331387 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.331364    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rg59\" (UniqueName: \"kubernetes.io/projected/0db55b6f-7977-4d46-9528-a8dcf74de6cd-kube-api-access-7rg59\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.331433 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.331415    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csrv2\" (UniqueName: \"kubernetes.io/projected/c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b-kube-api-access-csrv2\") pod \"network-check-source-7bc5b6bf85-4m4wn\" (UID: \"c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"
Jul 16 22:17:28.331475 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.331442    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a08215e-447d-4500-8136-3a196037bc48-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.331513 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.331500    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0db55b6f-7977-4d46-9528-a8dcf74de6cd-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.332661 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.332638    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.332802 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.332784    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-qcbpl\""
Jul 16 22:17:28.332862 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.332786    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.389543 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.389505    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"]
Jul 16 22:17:28.398787 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.398753    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"]
Jul 16 22:17:28.398937 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.398874    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:28.401249 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.401226    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 16 22:17:28.401249 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.401234    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.401428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.401371    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jsqn7\""
Jul 16 22:17:28.401473 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.401459    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.416815 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.416783    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-85f9cf987d-cs7ph"]
Jul 16 22:17:28.416966 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.416905    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.419355 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.419333    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.419477 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.419461    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-bx4wg\""
Jul 16 22:17:28.419525 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.419478    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.419525 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.419482    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 16 22:17:28.419603 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.419551    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 16 22:17:28.432172 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f96sm\" (UniqueName: \"kubernetes.io/projected/f36db1a6-eff5-45da-84a2-ca5d0572ac16-kube-api-access-f96sm\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.432300 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432177    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a08215e-447d-4500-8136-3a196037bc48-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.432300 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432199    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/75f6164d-4c29-4111-bdaf-961f00f83f12-snapshots\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.432300 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432217    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f36db1a6-eff5-45da-84a2-ca5d0572ac16-config\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.432418 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432353    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csrv2\" (UniqueName: \"kubernetes.io/projected/c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b-kube-api-access-csrv2\") pod \"network-check-source-7bc5b6bf85-4m4wn\" (UID: \"c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"
Jul 16 22:17:28.432418 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75f6164d-4c29-4111-bdaf-961f00f83f12-tmp\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.432486 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432414    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a08215e-447d-4500-8136-3a196037bc48-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.432524 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432507    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f6164d-4c29-4111-bdaf-961f00f83f12-service-ca-bundle\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.432563 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432536    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0db55b6f-7977-4d46-9528-a8dcf74de6cd-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.432610 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432564    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75f6164d-4c29-4111-bdaf-961f00f83f12-serving-cert\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.432610 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432598    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f6164d-4c29-4111-bdaf-961f00f83f12-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.432683 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432629    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhww\" (UniqueName: \"kubernetes.io/projected/4a08215e-447d-4500-8136-3a196037bc48-kube-api-access-twhww\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.432683 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432661    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f36db1a6-eff5-45da-84a2-ca5d0572ac16-serving-cert\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.432781 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432711    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cshk5\" (UniqueName: \"kubernetes.io/projected/89bd5ddb-b578-43b1-9bc2-1610f4d858c0-kube-api-access-cshk5\") pod \"volume-data-source-validator-78cf4c6f6-4qn8c\" (UID: \"89bd5ddb-b578-43b1-9bc2-1610f4d858c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"
Jul 16 22:17:28.432781 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432753    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a08215e-447d-4500-8136-3a196037bc48-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.432871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432781    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7ntx\" (UniqueName: \"kubernetes.io/projected/75f6164d-4c29-4111-bdaf-961f00f83f12-kube-api-access-f7ntx\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.432871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432801    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36db1a6-eff5-45da-84a2-ca5d0572ac16-trusted-ca\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.432871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.432871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.432861    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rg59\" (UniqueName: \"kubernetes.io/projected/0db55b6f-7977-4d46-9528-a8dcf74de6cd-kube-api-access-7rg59\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.433031 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.432918    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:28.433031 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.432964    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls podName:0db55b6f-7977-4d46-9528-a8dcf74de6cd nodeName:}" failed. No retries permitted until 2026-07-16 22:17:28.93295118 +0000 UTC m=+41.606210166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kfn8p" (UID: "0db55b6f-7977-4d46-9528-a8dcf74de6cd") : secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:28.433290 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.433273    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0db55b6f-7977-4d46-9528-a8dcf74de6cd-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.436239 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.436217    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a08215e-447d-4500-8136-3a196037bc48-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.437160 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.437143    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"]
Jul 16 22:17:28.437221 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.437176    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"]
Jul 16 22:17:28.437221 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.437190    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85f9cf987d-cs7ph"]
Jul 16 22:17:28.437221 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.437203    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.440581 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.440561    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 22:17:28.440780 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.440730    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 22:17:28.440874 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.440790    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 16 22:17:28.440874 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.440828    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c79bq\""
Jul 16 22:17:28.441079 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.441057    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 16 22:17:28.441171 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.441094    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 16 22:17:28.441455 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.441320    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 16 22:17:28.443582 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.443562    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twhww\" (UniqueName: \"kubernetes.io/projected/4a08215e-447d-4500-8136-3a196037bc48-kube-api-access-twhww\") pod \"kube-storage-version-migrator-operator-7658d9b749-7c55f\" (UID: \"4a08215e-447d-4500-8136-3a196037bc48\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.443968 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.443950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csrv2\" (UniqueName: \"kubernetes.io/projected/c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b-kube-api-access-csrv2\") pod \"network-check-source-7bc5b6bf85-4m4wn\" (UID: \"c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"
Jul 16 22:17:28.444069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.444049    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rg59\" (UniqueName: \"kubernetes.io/projected/0db55b6f-7977-4d46-9528-a8dcf74de6cd-kube-api-access-7rg59\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.511189 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.511144    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"
Jul 16 22:17:28.527021 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.526967    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"
Jul 16 22:17:28.533466 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533446    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r95fd\" (UniqueName: \"kubernetes.io/projected/387137d2-c707-406b-a263-8df8fd8d1d35-kube-api-access-r95fd\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.533574 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533475    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-stats-auth\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.533574 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533505    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f96sm\" (UniqueName: \"kubernetes.io/projected/f36db1a6-eff5-45da-84a2-ca5d0572ac16-kube-api-access-f96sm\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.533574 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533526    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-default-certificate\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.533745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533584    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.533745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/75f6164d-4c29-4111-bdaf-961f00f83f12-snapshots\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.533745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533679    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f36db1a6-eff5-45da-84a2-ca5d0572ac16-config\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.533867 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533763    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75f6164d-4c29-4111-bdaf-961f00f83f12-tmp\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.533867 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f6164d-4c29-4111-bdaf-961f00f83f12-service-ca-bundle\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.533867 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533831    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcqn9\" (UniqueName: \"kubernetes.io/projected/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-kube-api-access-tcqn9\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.533867 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533856    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/387137d2-c707-406b-a263-8df8fd8d1d35-config\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.534042 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533886    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75f6164d-4c29-4111-bdaf-961f00f83f12-serving-cert\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.534042 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f6164d-4c29-4111-bdaf-961f00f83f12-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.534042 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533954    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:28.534042 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.533975    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.534042 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534007    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcw6l\" (UniqueName: \"kubernetes.io/projected/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-kube-api-access-mcw6l\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:28.534042 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534027    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/387137d2-c707-406b-a263-8df8fd8d1d35-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.534342 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f36db1a6-eff5-45da-84a2-ca5d0572ac16-serving-cert\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.534342 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534074    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cshk5\" (UniqueName: \"kubernetes.io/projected/89bd5ddb-b578-43b1-9bc2-1610f4d858c0-kube-api-access-cshk5\") pod \"volume-data-source-validator-78cf4c6f6-4qn8c\" (UID: \"89bd5ddb-b578-43b1-9bc2-1610f4d858c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"
Jul 16 22:17:28.534342 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534099    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7ntx\" (UniqueName: \"kubernetes.io/projected/75f6164d-4c29-4111-bdaf-961f00f83f12-kube-api-access-f7ntx\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.534342 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534167    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36db1a6-eff5-45da-84a2-ca5d0572ac16-trusted-ca\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.534342 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534201    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75f6164d-4c29-4111-bdaf-961f00f83f12-tmp\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.534592 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534427    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f6164d-4c29-4111-bdaf-961f00f83f12-service-ca-bundle\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.534592 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534533    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f36db1a6-eff5-45da-84a2-ca5d0572ac16-config\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.534969 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534947    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/75f6164d-4c29-4111-bdaf-961f00f83f12-snapshots\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.535052 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.534997    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36db1a6-eff5-45da-84a2-ca5d0572ac16-trusted-ca\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.535531 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.535511    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f6164d-4c29-4111-bdaf-961f00f83f12-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.536379 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.536364    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75f6164d-4c29-4111-bdaf-961f00f83f12-serving-cert\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.536724 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.536702    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f36db1a6-eff5-45da-84a2-ca5d0572ac16-serving-cert\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.538537 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.538513    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:28.542074 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.542049    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f96sm\" (UniqueName: \"kubernetes.io/projected/f36db1a6-eff5-45da-84a2-ca5d0572ac16-kube-api-access-f96sm\") pod \"console-operator-565b7b569d-4b8xc\" (UID: \"f36db1a6-eff5-45da-84a2-ca5d0572ac16\") " pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.543043 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.543021    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7ntx\" (UniqueName: \"kubernetes.io/projected/75f6164d-4c29-4111-bdaf-961f00f83f12-kube-api-access-f7ntx\") pod \"insights-operator-555d9d7885-sf45x\" (UID: \"75f6164d-4c29-4111-bdaf-961f00f83f12\") " pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.543620 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.543595    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cshk5\" (UniqueName: \"kubernetes.io/projected/89bd5ddb-b578-43b1-9bc2-1610f4d858c0-kube-api-access-cshk5\") pod \"volume-data-source-validator-78cf4c6f6-4qn8c\" (UID: \"89bd5ddb-b578-43b1-9bc2-1610f4d858c0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"
Jul 16 22:17:28.567494 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.567461    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-sf45x"
Jul 16 22:17:28.619018 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.618731    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:28.635486 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcqn9\" (UniqueName: \"kubernetes.io/projected/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-kube-api-access-tcqn9\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.635486 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635448    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/387137d2-c707-406b-a263-8df8fd8d1d35-config\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.635738 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635624    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:28.635738 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635665    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.635738 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635730    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcw6l\" (UniqueName: \"kubernetes.io/projected/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-kube-api-access-mcw6l\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:28.635944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635757    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/387137d2-c707-406b-a263-8df8fd8d1d35-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.635944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635850    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r95fd\" (UniqueName: \"kubernetes.io/projected/387137d2-c707-406b-a263-8df8fd8d1d35-kube-api-access-r95fd\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.635944 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.635886    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:29.135864333 +0000 UTC m=+41.809123322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : configmap references non-existent config key: service-ca.crt
Jul 16 22:17:28.635944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-stats-auth\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.636177 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.635973    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-default-certificate\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.636177 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.636004    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.636177 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.636060    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/387137d2-c707-406b-a263-8df8fd8d1d35-config\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.636334 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.636291    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:17:28.636391 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.636365    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:29.136341581 +0000 UTC m=+41.809600581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : secret "router-metrics-certs-default" not found
Jul 16 22:17:28.636450 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.636436    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:17:28.636495 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.636474    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls podName:297a6aee-b2ad-4f18-9397-4ffc996c8f0f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:29.136462382 +0000 UTC m=+41.809721368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7c2pf" (UID: "297a6aee-b2ad-4f18-9397-4ffc996c8f0f") : secret "samples-operator-tls" not found
Jul 16 22:17:28.638482 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.638425    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"
Jul 16 22:17:28.638608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.638569    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/387137d2-c707-406b-a263-8df8fd8d1d35-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.639619 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.639597    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-default-certificate\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.639742 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.639664    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-stats-auth\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.645023 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.644735    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:28.650100 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.650068    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcqn9\" (UniqueName: \"kubernetes.io/projected/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-kube-api-access-tcqn9\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:28.650590 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.650566    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r95fd\" (UniqueName: \"kubernetes.io/projected/387137d2-c707-406b-a263-8df8fd8d1d35-kube-api-access-r95fd\") pod \"service-ca-operator-5bcbc94c78-8f4fw\" (UID: \"387137d2-c707-406b-a263-8df8fd8d1d35\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.652157 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.652097    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcw6l\" (UniqueName: \"kubernetes.io/projected/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-kube-api-access-mcw6l\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:28.721643 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.721321    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn"]
Jul 16 22:17:28.726480 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.726222    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"
Jul 16 22:17:28.755172 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.755130    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f"]
Jul 16 22:17:28.788156 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.787737    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sf45x"]
Jul 16 22:17:28.792121 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:28.792056    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75f6164d_4c29_4111_bdaf_961f00f83f12.slice/crio-8acc203eaff32cf7d325efb2db5ddfe8d474c4368036780871a8befce2ae9f83 WatchSource:0}: Error finding container 8acc203eaff32cf7d325efb2db5ddfe8d474c4368036780871a8befce2ae9f83: Status 404 returned error can't find the container with id 8acc203eaff32cf7d325efb2db5ddfe8d474c4368036780871a8befce2ae9f83
Jul 16 22:17:28.822974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.822926    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4b8xc"]
Jul 16 22:17:28.826137 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:28.826065    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36db1a6_eff5_45da_84a2_ca5d0572ac16.slice/crio-5cc145cc7daf673fceeb192b124c592bd244f2559e1578609771ae289af85e9a WatchSource:0}: Error finding container 5cc145cc7daf673fceeb192b124c592bd244f2559e1578609771ae289af85e9a: Status 404 returned error can't find the container with id 5cc145cc7daf673fceeb192b124c592bd244f2559e1578609771ae289af85e9a
Jul 16 22:17:28.859346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.859170    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c"]
Jul 16 22:17:28.862363 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:28.862338    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bd5ddb_b578_43b1_9bc2_1610f4d858c0.slice/crio-93f36c2f426654bdb964d6f46e180f58afe8fc1224971c6c0d08cce2e088858d WatchSource:0}: Error finding container 93f36c2f426654bdb964d6f46e180f58afe8fc1224971c6c0d08cce2e088858d: Status 404 returned error can't find the container with id 93f36c2f426654bdb964d6f46e180f58afe8fc1224971c6c0d08cce2e088858d
Jul 16 22:17:28.899092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.899059    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw"]
Jul 16 22:17:28.902200 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:28.902172    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387137d2_c707_406b_a263_8df8fd8d1d35.slice/crio-29cbadad84e08978d89ef47978a8b137ad23f4264352b7829e7c2de0158845b7 WatchSource:0}: Error finding container 29cbadad84e08978d89ef47978a8b137ad23f4264352b7829e7c2de0158845b7: Status 404 returned error can't find the container with id 29cbadad84e08978d89ef47978a8b137ad23f4264352b7829e7c2de0158845b7
Jul 16 22:17:28.939234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:28.939196    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:28.939408 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.939353    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:28.939464 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:28.939438    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls podName:0db55b6f-7977-4d46-9528-a8dcf74de6cd nodeName:}" failed. No retries permitted until 2026-07-16 22:17:29.939416482 +0000 UTC m=+42.612675482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kfn8p" (UID: "0db55b6f-7977-4d46-9528-a8dcf74de6cd") : secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:29.081194 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.081155    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cptpd" event={"ID":"4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584","Type":"ContainerStarted","Data":"807e3d49d94b60e73d4705fc35e22587487e538b42b645d9a92e21debb434e8c"}
Jul 16 22:17:29.082399 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.082372    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn" event={"ID":"c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b","Type":"ContainerStarted","Data":"f6f2189c01657ff86d1b1c33843b66d977c3e2cfc8f345874c73d1ecea079561"}
Jul 16 22:17:29.083513 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.083483    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw" event={"ID":"387137d2-c707-406b-a263-8df8fd8d1d35","Type":"ContainerStarted","Data":"29cbadad84e08978d89ef47978a8b137ad23f4264352b7829e7c2de0158845b7"}
Jul 16 22:17:29.084693 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.084663    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f" event={"ID":"4a08215e-447d-4500-8136-3a196037bc48","Type":"ContainerStarted","Data":"3c9a390b190ef114ee72c30e049daf4b999e2aea57b473298b126e97af42e00b"}
Jul 16 22:17:29.085865 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.085841    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c" event={"ID":"89bd5ddb-b578-43b1-9bc2-1610f4d858c0","Type":"ContainerStarted","Data":"93f36c2f426654bdb964d6f46e180f58afe8fc1224971c6c0d08cce2e088858d"}
Jul 16 22:17:29.086920 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.086900    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" event={"ID":"f36db1a6-eff5-45da-84a2-ca5d0572ac16","Type":"ContainerStarted","Data":"5cc145cc7daf673fceeb192b124c592bd244f2559e1578609771ae289af85e9a"}
Jul 16 22:17:29.087864 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.087841    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sf45x" event={"ID":"75f6164d-4c29-4111-bdaf-961f00f83f12","Type":"ContainerStarted","Data":"8acc203eaff32cf7d325efb2db5ddfe8d474c4368036780871a8befce2ae9f83"}
Jul 16 22:17:29.104877 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.104775    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cptpd" podStartSLOduration=7.092439948 podStartE2EDuration="42.104756286s" totalImagesPullingTime="35.012316338s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:29.103526056 +0000 UTC m=+41.776785084" watchObservedRunningTime="2026-07-16 22:17:29.104756286 +0000 UTC m=+41.778015295"
Jul 16 22:17:29.141458 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.141421    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:29.141604 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.141485    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:29.141604 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.141505    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:29.141604 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:29.141572    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:17:29.141742 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:29.141613    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:30.141599295 +0000 UTC m=+42.814858283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : configmap references non-existent config key: service-ca.crt
Jul 16 22:17:29.141742 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:29.141634    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:30.141620357 +0000 UTC m=+42.814879343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : secret "router-metrics-certs-default" not found
Jul 16 22:17:29.141742 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:29.141632    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:17:29.141742 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:29.141683    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls podName:297a6aee-b2ad-4f18-9397-4ffc996c8f0f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:30.141671993 +0000 UTC m=+42.814930980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7c2pf" (UID: "297a6aee-b2ad-4f18-9397-4ffc996c8f0f") : secret "samples-operator-tls" not found
Jul 16 22:17:29.227248 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.227208    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:29.299593 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.299559    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:29.950069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:29.949755    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:29.950069 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:29.950015    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:29.950309 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:29.950080    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls podName:0db55b6f-7977-4d46-9528-a8dcf74de6cd nodeName:}" failed. No retries permitted until 2026-07-16 22:17:31.950062172 +0000 UTC m=+44.623321164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kfn8p" (UID: "0db55b6f-7977-4d46-9528-a8dcf74de6cd") : secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:30.152282    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:30.152401    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:30.152432    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.152520    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.152577    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:32.152560797 +0000 UTC m=+44.825819806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : configmap references non-existent config key: service-ca.crt
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.152597    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:32.152587605 +0000 UTC m=+44.825846596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : secret "router-metrics-certs-default" not found
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.152620    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:17:30.152996 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.152649    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls podName:297a6aee-b2ad-4f18-9397-4ffc996c8f0f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:32.152639796 +0000 UTC m=+44.825898783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7c2pf" (UID: "297a6aee-b2ad-4f18-9397-4ffc996c8f0f") : secret "samples-operator-tls" not found
Jul 16 22:17:30.456851 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:30.455931    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:30.456851 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:30.456099    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:30.456851 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.456253    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 22:17:30.456851 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.456270    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779c67dc6c-q59db: secret "image-registry-tls" not found
Jul 16 22:17:30.456851 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.456343    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls podName:5f81de92-62c5-4b55-833d-1d1653130da3 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:38.456315448 +0000 UTC m=+51.129574437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls") pod "image-registry-779c67dc6c-q59db" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3") : secret "image-registry-tls" not found
Jul 16 22:17:30.456851 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.456749    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:17:30.456851 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:30.456799    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls podName:a704ee01-ae37-4011-bb07-0a1fd7ca87b9 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:38.45678249 +0000 UTC m=+51.130041479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls") pod "dns-default-2zxmt" (UID: "a704ee01-ae37-4011-bb07-0a1fd7ca87b9") : secret "dns-default-metrics-tls" not found
Jul 16 22:17:31.060677 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.059660    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"]
Jul 16 22:17:31.084026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.083995    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"]
Jul 16 22:17:31.084218 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.084150    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:31.087096 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.086549    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gt7tm\""
Jul 16 22:17:31.087096 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.086778    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 16 22:17:31.087096 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.086967    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 16 22:17:31.181586 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.181541    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:31.264508 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.264463    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:31.264706 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.264689    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:31.345157 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.345056    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:31.366171 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.365638    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:31.366171 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.365718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:31.366171 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:31.365856    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 22:17:31.366171 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:31.365937    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert podName:b3eec9f2-9166-4cdd-8a4c-01b227fddc76 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:31.865913106 +0000 UTC m=+44.539172095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-m8xj9" (UID: "b3eec9f2-9166-4cdd-8a4c-01b227fddc76") : secret "networking-console-plugin-cert" not found
Jul 16 22:17:31.366445 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.366251    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:31.447534 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.447500    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:31.870436 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.870402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:31.870643 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:31.870562    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 22:17:31.870643 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:31.870639    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert podName:b3eec9f2-9166-4cdd-8a4c-01b227fddc76 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:32.870619141 +0000 UTC m=+45.543878129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-m8xj9" (UID: "b3eec9f2-9166-4cdd-8a4c-01b227fddc76") : secret "networking-console-plugin-cert" not found
Jul 16 22:17:31.971227 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:31.971189    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:31.971490 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:31.971347    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:31.971490 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:31.971442    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls podName:0db55b6f-7977-4d46-9528-a8dcf74de6cd nodeName:}" failed. No retries permitted until 2026-07-16 22:17:35.971420442 +0000 UTC m=+48.644679443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kfn8p" (UID: "0db55b6f-7977-4d46-9528-a8dcf74de6cd") : secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:32.172060 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:32.171975    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:32.172060 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:32.172054    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:32.172289 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:32.172075    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:32.172289 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:32.172147    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:17:32.172289 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:32.172194    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:17:32.172289 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:32.172210    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:36.172196902 +0000 UTC m=+48.845455889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : configmap references non-existent config key: service-ca.crt
Jul 16 22:17:32.172289 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:32.172226    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:36.172219676 +0000 UTC m=+48.845478661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : secret "router-metrics-certs-default" not found
Jul 16 22:17:32.172289 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:32.172237    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls podName:297a6aee-b2ad-4f18-9397-4ffc996c8f0f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:36.172232324 +0000 UTC m=+48.845491310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7c2pf" (UID: "297a6aee-b2ad-4f18-9397-4ffc996c8f0f") : secret "samples-operator-tls" not found
Jul 16 22:17:32.876657 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:32.876610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:32.877163 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:32.876779    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 22:17:32.877163 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:32.876868    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert podName:b3eec9f2-9166-4cdd-8a4c-01b227fddc76 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:34.87684515 +0000 UTC m=+47.550104159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-m8xj9" (UID: "b3eec9f2-9166-4cdd-8a4c-01b227fddc76") : secret "networking-console-plugin-cert" not found
Jul 16 22:17:34.897498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:34.897455    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:34.897974 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:34.897588    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 22:17:34.897974 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:34.897654    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert podName:b3eec9f2-9166-4cdd-8a4c-01b227fddc76 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:38.897636448 +0000 UTC m=+51.570895437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-m8xj9" (UID: "b3eec9f2-9166-4cdd-8a4c-01b227fddc76") : secret "networking-console-plugin-cert" not found
Jul 16 22:17:36.007495 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.007438    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:36.007960 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:36.007619    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:36.007960 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:36.007705    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls podName:0db55b6f-7977-4d46-9528-a8dcf74de6cd nodeName:}" failed. No retries permitted until 2026-07-16 22:17:44.007683056 +0000 UTC m=+56.680942047 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kfn8p" (UID: "0db55b6f-7977-4d46-9528-a8dcf74de6cd") : secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:36.112625 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.112585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c" event={"ID":"89bd5ddb-b578-43b1-9bc2-1610f4d858c0","Type":"ContainerStarted","Data":"546e28a0b8492eab5ccc75bbc832b5227d324ba1929472144f0bce1717a6c73a"}
Jul 16 22:17:36.115427 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.115403    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/0.log"
Jul 16 22:17:36.115562 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.115449    2556 generic.go:356] "Generic (PLEG): container finished" podID="f36db1a6-eff5-45da-84a2-ca5d0572ac16" containerID="1080d7f4d1ce5c7b477796bc32e718c278802d285a3b28066b8c92a4977b1a31" exitCode=255
Jul 16 22:17:36.115562 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.115520    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" event={"ID":"f36db1a6-eff5-45da-84a2-ca5d0572ac16","Type":"ContainerDied","Data":"1080d7f4d1ce5c7b477796bc32e718c278802d285a3b28066b8c92a4977b1a31"}
Jul 16 22:17:36.115732 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.115716    2556 scope.go:117] "RemoveContainer" containerID="1080d7f4d1ce5c7b477796bc32e718c278802d285a3b28066b8c92a4977b1a31"
Jul 16 22:17:36.119334 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.119305    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sf45x" event={"ID":"75f6164d-4c29-4111-bdaf-961f00f83f12","Type":"ContainerStarted","Data":"48699b41ed91f98c36c443d2ee5e2e995032bf9557425b7248b0722cc8862c4b"}
Jul 16 22:17:36.120993 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.120941    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn" event={"ID":"c1ff1b24-12c5-42ff-bfb8-3f97c0e8d10b","Type":"ContainerStarted","Data":"77524c3c1eb0d37079328394f6ca0703d9135eb439b97ea8ae8c693b9835ea19"}
Jul 16 22:17:36.122489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.122456    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw" event={"ID":"387137d2-c707-406b-a263-8df8fd8d1d35","Type":"ContainerStarted","Data":"c9a77e95b66876a230d801796ca0ea13bf9a72e7392bdf9b97cefb81e842f988"}
Jul 16 22:17:36.124315 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.124295    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f" event={"ID":"4a08215e-447d-4500-8136-3a196037bc48","Type":"ContainerStarted","Data":"c0030309202becade18743282b2b7d1ec2516abf63e02dae19265c5cd8677cec"}
Jul 16 22:17:36.128068 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.127677    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4qn8c" podStartSLOduration=1.718316738 podStartE2EDuration="8.127662353s" totalImagesPullingTime="6.409345615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:36.127456504 +0000 UTC m=+48.800715538" watchObservedRunningTime="2026-07-16 22:17:36.127662353 +0000 UTC m=+48.800921362"
Jul 16 22:17:36.147277 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.147225    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f" podStartSLOduration=1.621979138 podStartE2EDuration="8.14720586s" totalImagesPullingTime="6.525226722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:36.145850219 +0000 UTC m=+48.819109252" watchObservedRunningTime="2026-07-16 22:17:36.14720586 +0000 UTC m=+48.820464868"
Jul 16 22:17:36.178966 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.178542    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw" podStartSLOduration=1.780980004 podStartE2EDuration="8.178521108s" totalImagesPullingTime="6.397541104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:36.177317482 +0000 UTC m=+48.850576490" watchObservedRunningTime="2026-07-16 22:17:36.178521108 +0000 UTC m=+48.851780118"
Jul 16 22:17:36.208724 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.208688    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:36.211617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.211590    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:36.212479 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.212450    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:36.212626 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:36.211209    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:17:36.212783 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:36.212388    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:17:36.212916 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:36.212714    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:44.212693279 +0000 UTC m=+56.885952267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : configmap references non-existent config key: service-ca.crt
Jul 16 22:17:36.213062 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:36.213039    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:44.213016114 +0000 UTC m=+56.886275103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : secret "router-metrics-certs-default" not found
Jul 16 22:17:36.213211 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:36.213071    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls podName:297a6aee-b2ad-4f18-9397-4ffc996c8f0f nodeName:}" failed. No retries permitted until 2026-07-16 22:17:44.213060241 +0000 UTC m=+56.886319227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7c2pf" (UID: "297a6aee-b2ad-4f18-9397-4ffc996c8f0f") : secret "samples-operator-tls" not found
Jul 16 22:17:36.219456 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.219402    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4m4wn" podStartSLOduration=1.362678424 podStartE2EDuration="8.219381879s" totalImagesPullingTime="6.856703455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:36.218723821 +0000 UTC m=+48.891982832" watchObservedRunningTime="2026-07-16 22:17:36.219381879 +0000 UTC m=+48.892640887"
Jul 16 22:17:36.219832 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.219794    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-sf45x" podStartSLOduration=1.7215924089999999 podStartE2EDuration="8.219782219s" totalImagesPullingTime="6.49818981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:36.199263569 +0000 UTC m=+48.872522578" watchObservedRunningTime="2026-07-16 22:17:36.219782219 +0000 UTC m=+48.893041228"
Jul 16 22:17:36.232129 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.231476    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:36.291570 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.291536    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:36.944235 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:36.944194    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:37.027934 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.027895    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-l4qs4"]
Jul 16 22:17:37.042471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.042433    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.045736 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.045715    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 22:17:37.046278 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.046236    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 22:17:37.052986 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.052966    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cgkc2\""
Jul 16 22:17:37.053159 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.053139    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l4qs4"]
Jul 16 22:17:37.105538 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.105486    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:37.121618 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.121587    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twsd9\" (UniqueName: \"kubernetes.io/projected/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-api-access-twsd9\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.121782 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.121668    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.121825 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.121791    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be6d1ea1-7566-4a06-bca8-47d0a98525de-crio-socket\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.121868 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.121826    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be6d1ea1-7566-4a06-bca8-47d0a98525de-data-volume\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.121954 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.121936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.130020 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.129998    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/1.log"
Jul 16 22:17:37.130446 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.130430    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/0.log"
Jul 16 22:17:37.130509 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.130472    2556 generic.go:356] "Generic (PLEG): container finished" podID="f36db1a6-eff5-45da-84a2-ca5d0572ac16" containerID="6958603f4e00798799f8b503809c4ca450bf803f2754fd5a897c3aa1f8748f46" exitCode=255
Jul 16 22:17:37.130584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.130559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" event={"ID":"f36db1a6-eff5-45da-84a2-ca5d0572ac16","Type":"ContainerDied","Data":"6958603f4e00798799f8b503809c4ca450bf803f2754fd5a897c3aa1f8748f46"}
Jul 16 22:17:37.130662 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.130627    2556 scope.go:117] "RemoveContainer" containerID="1080d7f4d1ce5c7b477796bc32e718c278802d285a3b28066b8c92a4977b1a31"
Jul 16 22:17:37.130918 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.130902    2556 scope.go:117] "RemoveContainer" containerID="6958603f4e00798799f8b503809c4ca450bf803f2754fd5a897c3aa1f8748f46"
Jul 16 22:17:37.131092 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.131068    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4b8xc_openshift-console-operator(f36db1a6-eff5-45da-84a2-ca5d0572ac16)\"" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" podUID="f36db1a6-eff5-45da-84a2-ca5d0572ac16"
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.223526    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.223606    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.223658    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be6d1ea1-7566-4a06-bca8-47d0a98525de-crio-socket\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.223674    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls podName:be6d1ea1-7566-4a06-bca8-47d0a98525de nodeName:}" failed. No retries permitted until 2026-07-16 22:17:37.723653758 +0000 UTC m=+50.396912744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4qs4" (UID: "be6d1ea1-7566-4a06-bca8-47d0a98525de") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.223707    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be6d1ea1-7566-4a06-bca8-47d0a98525de-data-volume\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.223831    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.223844    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be6d1ea1-7566-4a06-bca8-47d0a98525de-crio-socket\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.223897    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.223942    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm podName:be6d1ea1-7566-4a06-bca8-47d0a98525de nodeName:}" failed. No retries permitted until 2026-07-16 22:17:37.723926844 +0000 UTC m=+50.397185831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4qs4" (UID: "be6d1ea1-7566-4a06-bca8-47d0a98525de") : configmap "kube-rbac-proxy" not found
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.223976    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twsd9\" (UniqueName: \"kubernetes.io/projected/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-api-access-twsd9\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.224153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.224028    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be6d1ea1-7566-4a06-bca8-47d0a98525de-data-volume\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.236911 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.236880    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twsd9\" (UniqueName: \"kubernetes.io/projected/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-api-access-twsd9\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.301869 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.301829    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:37.302350 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.302329    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"]
Jul 16 22:17:37.323017 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.322988    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"]
Jul 16 22:17:37.323257 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.323143    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"
Jul 16 22:17:37.325430 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.325404    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9cm6g\""
Jul 16 22:17:37.325430 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.325422    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 16 22:17:37.325582 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.325438    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 16 22:17:37.365146 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.365091    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:37.425762 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.425725    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxp8d\" (UniqueName: \"kubernetes.io/projected/1b785b66-f8f6-4280-b573-cee4f2694443-kube-api-access-kxp8d\") pod \"migrator-7cc9c7ffd7-6g5zn\" (UID: \"1b785b66-f8f6-4280-b573-cee4f2694443\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"
Jul 16 22:17:37.466400 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.466364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:37.527079 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.526980    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxp8d\" (UniqueName: \"kubernetes.io/projected/1b785b66-f8f6-4280-b573-cee4f2694443-kube-api-access-kxp8d\") pod \"migrator-7cc9c7ffd7-6g5zn\" (UID: \"1b785b66-f8f6-4280-b573-cee4f2694443\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"
Jul 16 22:17:37.535244 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.535214    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxp8d\" (UniqueName: \"kubernetes.io/projected/1b785b66-f8f6-4280-b573-cee4f2694443-kube-api-access-kxp8d\") pod \"migrator-7cc9c7ffd7-6g5zn\" (UID: \"1b785b66-f8f6-4280-b573-cee4f2694443\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"
Jul 16 22:17:37.633576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.633536    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"
Jul 16 22:17:37.652369 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.652331    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:37.728945 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.728907    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.729102 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.729004    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:37.729102 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.729025    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 22:17:37.729102 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.729083    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm podName:be6d1ea1-7566-4a06-bca8-47d0a98525de nodeName:}" failed. No retries permitted until 2026-07-16 22:17:38.729066189 +0000 UTC m=+51.402325175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4qs4" (UID: "be6d1ea1-7566-4a06-bca8-47d0a98525de") : configmap "kube-rbac-proxy" not found
Jul 16 22:17:37.729282 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.729145    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:37.729282 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:37.729184    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls podName:be6d1ea1-7566-4a06-bca8-47d0a98525de nodeName:}" failed. No retries permitted until 2026-07-16 22:17:38.729173333 +0000 UTC m=+51.402432322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4qs4" (UID: "be6d1ea1-7566-4a06-bca8-47d0a98525de") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:37.783261 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:37.783189    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn"]
Jul 16 22:17:37.786315 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:37.786282    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b785b66_f8f6_4280_b573_cee4f2694443.slice/crio-dbefa7817a5ea45f2d520d1f1902e06bbdc0400f0a45939fec4188c8c101e354 WatchSource:0}: Error finding container dbefa7817a5ea45f2d520d1f1902e06bbdc0400f0a45939fec4188c8c101e354: Status 404 returned error can't find the container with id dbefa7817a5ea45f2d520d1f1902e06bbdc0400f0a45939fec4188c8c101e354
Jul 16 22:17:38.134753 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.134712    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn" event={"ID":"1b785b66-f8f6-4280-b573-cee4f2694443","Type":"ContainerStarted","Data":"dbefa7817a5ea45f2d520d1f1902e06bbdc0400f0a45939fec4188c8c101e354"}
Jul 16 22:17:38.136234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.136215    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/1.log"
Jul 16 22:17:38.136566 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.136549    2556 scope.go:117] "RemoveContainer" containerID="6958603f4e00798799f8b503809c4ca450bf803f2754fd5a897c3aa1f8748f46"
Jul 16 22:17:38.136724 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.136707    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4b8xc_openshift-console-operator(f36db1a6-eff5-45da-84a2-ca5d0572ac16)\"" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" podUID="f36db1a6-eff5-45da-84a2-ca5d0572ac16"
Jul 16 22:17:38.217400 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.217372    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7nft5_fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3/dns-node-resolver/0.log"
Jul 16 22:17:38.262558 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.262514    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:38.536471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.536428    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:38.536677 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.536555    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:38.536677 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.536597    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 22:17:38.536677 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.536622    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779c67dc6c-q59db: secret "image-registry-tls" not found
Jul 16 22:17:38.536848 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.536689    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:17:38.536848 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.536693    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls podName:5f81de92-62c5-4b55-833d-1d1653130da3 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:54.536669887 +0000 UTC m=+67.209928878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls") pod "image-registry-779c67dc6c-q59db" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3") : secret "image-registry-tls" not found
Jul 16 22:17:38.536848 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.536756    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls podName:a704ee01-ae37-4011-bb07-0a1fd7ca87b9 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:54.536744186 +0000 UTC m=+67.210003175 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls") pod "dns-default-2zxmt" (UID: "a704ee01-ae37-4011-bb07-0a1fd7ca87b9") : secret "dns-default-metrics-tls" not found
Jul 16 22:17:38.616294 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.616262    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:38.619995 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.619966    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:38.620181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.620028    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:38.739017 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.738974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:38.739220 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.739150    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:38.739354 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.739320    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:38.739445 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.739431    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls podName:be6d1ea1-7566-4a06-bca8-47d0a98525de nodeName:}" failed. No retries permitted until 2026-07-16 22:17:40.73941009 +0000 UTC m=+53.412669078 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4qs4" (UID: "be6d1ea1-7566-4a06-bca8-47d0a98525de") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:38.739599 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.739573    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be6d1ea1-7566-4a06-bca8-47d0a98525de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:38.941569 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:38.941479    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:38.941711 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.941638    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 22:17:38.941711 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:38.941704    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert podName:b3eec9f2-9166-4cdd-8a4c-01b227fddc76 nodeName:}" failed. No retries permitted until 2026-07-16 22:17:46.941684842 +0000 UTC m=+59.614943842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-m8xj9" (UID: "b3eec9f2-9166-4cdd-8a4c-01b227fddc76") : secret "networking-console-plugin-cert" not found
Jul 16 22:17:39.141840 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.141810    2556 scope.go:117] "RemoveContainer" containerID="6958603f4e00798799f8b503809c4ca450bf803f2754fd5a897c3aa1f8748f46"
Jul 16 22:17:39.142315 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:39.142052    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4b8xc_openshift-console-operator(f36db1a6-eff5-45da-84a2-ca5d0572ac16)\"" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" podUID="f36db1a6-eff5-45da-84a2-ca5d0572ac16"
Jul 16 22:17:39.218280 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.218206    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8gvtz_4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7/node-ca/0.log"
Jul 16 22:17:39.248848 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.248810    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-n6qz8"]
Jul 16 22:17:39.269830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.269789    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-n6qz8"]
Jul 16 22:17:39.269990 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.269885    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.272572 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.272547    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 16 22:17:39.272725 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.272626    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 16 22:17:39.272725 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.272666    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 16 22:17:39.272945 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.272929    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 16 22:17:39.273075 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.273056    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-297g8\""
Jul 16 22:17:39.346319 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.346277    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lc89\" (UniqueName: \"kubernetes.io/projected/6d32565a-3931-4394-bcf3-1d5004cc074a-kube-api-access-6lc89\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.346488 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.346326    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6d32565a-3931-4394-bcf3-1d5004cc074a-signing-cabundle\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.346488 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.346429    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6d32565a-3931-4394-bcf3-1d5004cc074a-signing-key\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.447179 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.447139    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6d32565a-3931-4394-bcf3-1d5004cc074a-signing-key\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.447363 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.447273    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lc89\" (UniqueName: \"kubernetes.io/projected/6d32565a-3931-4394-bcf3-1d5004cc074a-kube-api-access-6lc89\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.447363 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.447300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6d32565a-3931-4394-bcf3-1d5004cc074a-signing-cabundle\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.447919 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.447888    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6d32565a-3931-4394-bcf3-1d5004cc074a-signing-cabundle\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.449731 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.449708    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6d32565a-3931-4394-bcf3-1d5004cc074a-signing-key\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.456091 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.456066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lc89\" (UniqueName: \"kubernetes.io/projected/6d32565a-3931-4394-bcf3-1d5004cc074a-kube-api-access-6lc89\") pod \"service-ca-58598695f7-n6qz8\" (UID: \"6d32565a-3931-4394-bcf3-1d5004cc074a\") " pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.579192 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.579158    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-n6qz8"
Jul 16 22:17:39.825710 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.825680    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-49m5t_13f7c116-08cb-4fe3-9c54-ab12fb15b424/serve-healthcheck-canary/0.log"
Jul 16 22:17:39.849859 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.849828    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-n6qz8"]
Jul 16 22:17:39.853333 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:39.853310    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d32565a_3931_4394_bcf3_1d5004cc074a.slice/crio-fea273e9a8cb2d58767b7e272d246959a3afe63f69984267ac661bd6c851de01 WatchSource:0}: Error finding container fea273e9a8cb2d58767b7e272d246959a3afe63f69984267ac661bd6c851de01: Status 404 returned error can't find the container with id fea273e9a8cb2d58767b7e272d246959a3afe63f69984267ac661bd6c851de01
Jul 16 22:17:39.893618 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:39.893586    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:40.146051 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.145957    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-n6qz8" event={"ID":"6d32565a-3931-4394-bcf3-1d5004cc074a","Type":"ContainerStarted","Data":"3b143c3b73566d27609cb0a3ed498e9b9a7c41d8e5535d8f7d80401b1b1ec3c2"}
Jul 16 22:17:40.146051 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.146014    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-n6qz8" event={"ID":"6d32565a-3931-4394-bcf3-1d5004cc074a","Type":"ContainerStarted","Data":"fea273e9a8cb2d58767b7e272d246959a3afe63f69984267ac661bd6c851de01"}
Jul 16 22:17:40.147860 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.147834    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn" event={"ID":"1b785b66-f8f6-4280-b573-cee4f2694443","Type":"ContainerStarted","Data":"d0d5f18e3744375cd56991c42969063d1fcbecc715bda8d8868f6ccedf3c7943"}
Jul 16 22:17:40.147972 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.147876    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn" event={"ID":"1b785b66-f8f6-4280-b573-cee4f2694443","Type":"ContainerStarted","Data":"bd4135d026bf37e74b900a51e73305139f2982861ddd4b5f289f19116c93ac4c"}
Jul 16 22:17:40.194345 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.194276    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-n6qz8" podStartSLOduration=1.194251992 podStartE2EDuration="1.194251992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:40.169411521 +0000 UTC m=+52.842670528" watchObservedRunningTime="2026-07-16 22:17:40.194251992 +0000 UTC m=+52.867511001"
Jul 16 22:17:40.194776 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.194745    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6g5zn" podStartSLOduration=1.271781547 podStartE2EDuration="3.194734956s" totalImagesPullingTime="1.922953409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:40.194440882 +0000 UTC m=+52.867699881" watchObservedRunningTime="2026-07-16 22:17:40.194734956 +0000 UTC m=+52.867993963"
Jul 16 22:17:40.205312 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.205277    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:40.760084 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:40.760042    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:40.760286 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:40.760218    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:40.760336 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:40.760290    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls podName:be6d1ea1-7566-4a06-bca8-47d0a98525de nodeName:}" failed. No retries permitted until 2026-07-16 22:17:44.760272566 +0000 UTC m=+57.433531553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4qs4" (UID: "be6d1ea1-7566-4a06-bca8-47d0a98525de") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:41.734742 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:41.734706    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:41.839784 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:41.839750    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:42.081232 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:42.081199    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:42.183286 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:42.183250    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:42.345326 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:42.345240    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:42.453082 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:42.453023    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:42.554423 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:42.554384    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:42.667100 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:42.667009    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:42.778104 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:42.778070    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:43.359424 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:43.359386    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:43.587311 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:43.587276    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:44.090491 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.090446    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:17:44.090881 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:44.090609    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:44.090881 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:44.090692    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls podName:0db55b6f-7977-4d46-9528-a8dcf74de6cd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:00.090675626 +0000 UTC m=+72.763934612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-kfn8p" (UID: "0db55b6f-7977-4d46-9528-a8dcf74de6cd") : secret "cluster-monitoring-operator-tls" not found
Jul 16 22:17:44.179556 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.179514    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:44.292284 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.292242    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:44.292284 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.292283    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:44.292507 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:44.292451    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:00.292430006 +0000 UTC m=+72.965688992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : configmap references non-existent config key: service-ca.crt
Jul 16 22:17:44.292564 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.292520    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:17:44.292690 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:44.292672    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:17:44.292736 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:44.292727    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs podName:ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:00.292714639 +0000 UTC m=+72.965973625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs") pod "router-default-85f9cf987d-cs7ph" (UID: "ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072") : secret "router-metrics-certs-default" not found
Jul 16 22:17:44.294896 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.294872    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/297a6aee-b2ad-4f18-9397-4ffc996c8f0f-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7c2pf\" (UID: \"297a6aee-b2ad-4f18-9397-4ffc996c8f0f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:44.308051 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.308023    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"
Jul 16 22:17:44.384382 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.384347    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:44.438974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.438798    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf"]
Jul 16 22:17:44.796852 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:44.796819    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:44.797018 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:44.796932    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:44.797018 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:44.796984    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls podName:be6d1ea1-7566-4a06-bca8-47d0a98525de nodeName:}" failed. No retries permitted until 2026-07-16 22:17:52.796971269 +0000 UTC m=+65.470230254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4qs4" (UID: "be6d1ea1-7566-4a06-bca8-47d0a98525de") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:17:45.167244 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:45.167153    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf" event={"ID":"297a6aee-b2ad-4f18-9397-4ffc996c8f0f","Type":"ContainerStarted","Data":"bb3fa708a7dc3f44b19a340a42466eec37c52545267a99fcf34e4a7919258644"}
Jul 16 22:17:45.330814 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:45.330769    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:45.431481 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:45.431397    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:45.590974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:45.590922    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:45.778195 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:45.778088    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:46.100847 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:46.100804    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:46.202352 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:46.202314    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:46.374182 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:46.374079    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:46.615302 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:46.615281    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:46.826559 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:46.826510    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:46.885318 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:46.885243    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:47.019069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:47.019034    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:17:47.019272 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:47.019204    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 22:17:47.019316 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:47.019276    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert podName:b3eec9f2-9166-4cdd-8a4c-01b227fddc76 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:03.019252529 +0000 UTC m=+75.692511517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-m8xj9" (UID: "b3eec9f2-9166-4cdd-8a4c-01b227fddc76") : secret "networking-console-plugin-cert" not found
Jul 16 22:17:47.175947 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:47.175865    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf" event={"ID":"297a6aee-b2ad-4f18-9397-4ffc996c8f0f","Type":"ContainerStarted","Data":"3d46bd9e8eef8682e5507913cc77b35156910d36694c7bd225e1fa255f874d74"}
Jul 16 22:17:47.175947 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:47.175917    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf" event={"ID":"297a6aee-b2ad-4f18-9397-4ffc996c8f0f","Type":"ContainerStarted","Data":"23e82f7406e90359d53625ecba3bb2468724026ce663a855b8329f250d3d3013"}
Jul 16 22:17:47.194913 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:47.194870    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7c2pf" podStartSLOduration=17.116559195 podStartE2EDuration="19.194855754s" totalImagesPullingTime="2.078296559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:47.193590824 +0000 UTC m=+59.866849832" watchObservedRunningTime="2026-07-16 22:17:47.194855754 +0000 UTC m=+59.868114740"
Jul 16 22:17:47.978010 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:47.977970    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:48.048599 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:48.048572    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8k4z4"
Jul 16 22:17:48.177228 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:48.177192    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:48.393958 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:48.393915    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:48.494568 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:48.494530    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:48.592431 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:48.592386    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:48.773310 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:48.773224    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:49.787507 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:49.787475    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:49.806027 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:49.806010    2556 scope.go:117] "RemoveContainer" containerID="6958603f4e00798799f8b503809c4ca450bf803f2754fd5a897c3aa1f8748f46"
Jul 16 22:17:49.983576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:49.983540    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:50.190791 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.190717    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:17:50.191165 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.191147    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/1.log"
Jul 16 22:17:50.191279 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.191185    2556 generic.go:356] "Generic (PLEG): container finished" podID="f36db1a6-eff5-45da-84a2-ca5d0572ac16" containerID="555329c19ef6d86c0f7dd459d276ad1399b79de40dfe34cc86aec8d2aebfe33d" exitCode=255
Jul 16 22:17:50.191279 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.191261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" event={"ID":"f36db1a6-eff5-45da-84a2-ca5d0572ac16","Type":"ContainerDied","Data":"555329c19ef6d86c0f7dd459d276ad1399b79de40dfe34cc86aec8d2aebfe33d"}
Jul 16 22:17:50.192395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.191337    2556 scope.go:117] "RemoveContainer" containerID="6958603f4e00798799f8b503809c4ca450bf803f2754fd5a897c3aa1f8748f46"
Jul 16 22:17:50.192471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.192458    2556 scope.go:117] "RemoveContainer" containerID="555329c19ef6d86c0f7dd459d276ad1399b79de40dfe34cc86aec8d2aebfe33d"
Jul 16 22:17:50.192677 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:50.192656    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-4b8xc_openshift-console-operator(f36db1a6-eff5-45da-84a2-ca5d0572ac16)\"" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" podUID="f36db1a6-eff5-45da-84a2-ca5d0572ac16"
Jul 16 22:17:50.378371 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.378331    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:50.575397 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.575357    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:50.988612 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:50.988509    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:51.196495 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:51.196468    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:17:51.574717 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:51.574676    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:51.733576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:51.733538    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:51.972175 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:51.972042    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:52.373585 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.373549    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:52.471037 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.471001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:52.473490 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.473471    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 22:17:52.484484 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.484465    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b45c42e2-acce-454f-8aa5-59e0b573e914-metrics-certs\") pod \"network-metrics-daemon-fd92h\" (UID: \"b45c42e2-acce-454f-8aa5-59e0b573e914\") " pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:52.532910 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.532882    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjst2\""
Jul 16 22:17:52.541810 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.541786    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fd92h"
Jul 16 22:17:52.572596 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.572563    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:52.574935 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.574915    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 22:17:52.579365 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.579333    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:52.585269 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.585247    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c4d9b940-f340-4329-aea0-6252d6d331d0-original-pull-secret\") pod \"global-pull-secret-syncer-hhz6k\" (UID: \"c4d9b940-f340-4329-aea0-6252d6d331d0\") " pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:52.669944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.669776    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fd92h"]
Jul 16 22:17:52.672340 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:52.672310    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45c42e2_acce_454f_8aa5_59e0b573e914.slice/crio-9fb16e7e8d4dc604eadb59cf2a8865ca8dd1341d00af1e1b8ab4300785693ece WatchSource:0}: Error finding container 9fb16e7e8d4dc604eadb59cf2a8865ca8dd1341d00af1e1b8ab4300785693ece: Status 404 returned error can't find the container with id 9fb16e7e8d4dc604eadb59cf2a8865ca8dd1341d00af1e1b8ab4300785693ece
Jul 16 22:17:52.673402 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.673376    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:52.676362 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.676336    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n28z8\" (UniqueName: \"kubernetes.io/projected/50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f-kube-api-access-n28z8\") pod \"network-check-target-52pn4\" (UID: \"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f\") " pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:52.770736 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.770695    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:52.816993 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.816963    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhz6k"
Jul 16 22:17:52.826788 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.826769    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqtwh\""
Jul 16 22:17:52.834726 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.834701    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:52.875143 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.875087    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:52.878078 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.878056    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be6d1ea1-7566-4a06-bca8-47d0a98525de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4qs4\" (UID: \"be6d1ea1-7566-4a06-bca8-47d0a98525de\") " pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:52.955675 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.955646    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cgkc2\""
Jul 16 22:17:52.963463 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.963442    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l4qs4"
Jul 16 22:17:52.970715 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.970691    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhz6k"]
Jul 16 22:17:52.974005 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:52.973979    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d9b940_f340_4329_aea0_6252d6d331d0.slice/crio-4378842a0ed1ac6313435a97f3918528f5b526719c11dda214fff06ca9e98e85 WatchSource:0}: Error finding container 4378842a0ed1ac6313435a97f3918528f5b526719c11dda214fff06ca9e98e85: Status 404 returned error can't find the container with id 4378842a0ed1ac6313435a97f3918528f5b526719c11dda214fff06ca9e98e85
Jul 16 22:17:52.987357 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:52.987258    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-52pn4"]
Jul 16 22:17:52.989747 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:52.989717    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50bd9d6b_fcb3_4d9d_83b4_6698b026ea8f.slice/crio-0311b16cc38d367db370568e502f269ec23c2d3383c16378b03dc76fb84f1647 WatchSource:0}: Error finding container 0311b16cc38d367db370568e502f269ec23c2d3383c16378b03dc76fb84f1647: Status 404 returned error can't find the container with id 0311b16cc38d367db370568e502f269ec23c2d3383c16378b03dc76fb84f1647
Jul 16 22:17:53.094731 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.094702    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l4qs4"]
Jul 16 22:17:53.097687 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:17:53.097663    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe6d1ea1_7566_4a06_bca8_47d0a98525de.slice/crio-552045f7abbf004673daf9f8904f0b9252db0841bfa7d53645afd397a126935a WatchSource:0}: Error finding container 552045f7abbf004673daf9f8904f0b9252db0841bfa7d53645afd397a126935a: Status 404 returned error can't find the container with id 552045f7abbf004673daf9f8904f0b9252db0841bfa7d53645afd397a126935a
Jul 16 22:17:53.203305 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.203216    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4qs4" event={"ID":"be6d1ea1-7566-4a06-bca8-47d0a98525de","Type":"ContainerStarted","Data":"552045f7abbf004673daf9f8904f0b9252db0841bfa7d53645afd397a126935a"}
Jul 16 22:17:53.204378 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.204350    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fd92h" event={"ID":"b45c42e2-acce-454f-8aa5-59e0b573e914","Type":"ContainerStarted","Data":"9fb16e7e8d4dc604eadb59cf2a8865ca8dd1341d00af1e1b8ab4300785693ece"}
Jul 16 22:17:53.205795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.205767    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52pn4" event={"ID":"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f","Type":"ContainerStarted","Data":"36cd69afb5659698b41b71c8e6a50b93b51b5d6d52d73a44e735ca2e3bb9dc46"}
Jul 16 22:17:53.205868 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.205804    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-52pn4" event={"ID":"50bd9d6b-fcb3-4d9d-83b4-6698b026ea8f","Type":"ContainerStarted","Data":"0311b16cc38d367db370568e502f269ec23c2d3383c16378b03dc76fb84f1647"}
Jul 16 22:17:53.205868 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.205831    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:17:53.206872 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.206853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhz6k" event={"ID":"c4d9b940-f340-4329-aea0-6252d6d331d0","Type":"ContainerStarted","Data":"4378842a0ed1ac6313435a97f3918528f5b526719c11dda214fff06ca9e98e85"}
Jul 16 22:17:53.222093 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:53.222052    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-52pn4" podStartSLOduration=65.222040102 podStartE2EDuration="1m5.222040102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:53.220981832 +0000 UTC m=+65.894240841" watchObservedRunningTime="2026-07-16 22:17:53.222040102 +0000 UTC m=+65.895299110"
Jul 16 22:17:54.592031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.591153    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:54.592031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.591286    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:54.594450 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.594378    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a704ee01-ae37-4011-bb07-0a1fd7ca87b9-metrics-tls\") pod \"dns-default-2zxmt\" (UID: \"a704ee01-ae37-4011-bb07-0a1fd7ca87b9\") " pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:54.595057 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.595015    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"image-registry-779c67dc6c-q59db\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") " pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:54.791468 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.791431    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cmptb\""
Jul 16 22:17:54.799626 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.799595    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:54.803745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.803721    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4hsw2\""
Jul 16 22:17:54.811970 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.811930    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:17:54.968209 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.968184    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779c67dc6c-q59db"]
Jul 16 22:17:54.989900 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:54.989814    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zxmt"]
Jul 16 22:17:55.219341 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.218705    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779c67dc6c-q59db" event={"ID":"5f81de92-62c5-4b55-833d-1d1653130da3","Type":"ContainerStarted","Data":"58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82"}
Jul 16 22:17:55.219341 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.218777    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:17:55.219341 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.218800    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779c67dc6c-q59db" event={"ID":"5f81de92-62c5-4b55-833d-1d1653130da3","Type":"ContainerStarted","Data":"c4073f4b470f4d46d94fa416927571fa48112877fc6f081522de9be708469069"}
Jul 16 22:17:55.221576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.221485    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fd92h" event={"ID":"b45c42e2-acce-454f-8aa5-59e0b573e914","Type":"ContainerStarted","Data":"7956202fb716c7dddacf2d98c56c9984b7c24b7be9b5de67252a8b4316049f64"}
Jul 16 22:17:55.221576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.221542    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fd92h" event={"ID":"b45c42e2-acce-454f-8aa5-59e0b573e914","Type":"ContainerStarted","Data":"3092fa04d2b1588d212d197ebc761a2b2edd53d44276f344d99dc6095dd155a8"}
Jul 16 22:17:55.223782 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.223744    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zxmt" event={"ID":"a704ee01-ae37-4011-bb07-0a1fd7ca87b9","Type":"ContainerStarted","Data":"f70e8427ff8a4ae288fe6e73fab2e386d6f95d3d703025be5b759c7fab62fdd7"}
Jul 16 22:17:55.255445 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.255394    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fd92h" podStartSLOduration=66.571488841 podStartE2EDuration="1m8.255377663s" totalImagesPullingTime="1.683888822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:55.254667746 +0000 UTC m=+67.927926808" watchObservedRunningTime="2026-07-16 22:17:55.255377663 +0000 UTC m=+67.928636670"
Jul 16 22:17:55.256538 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:55.256492    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-779c67dc6c-q59db" podStartSLOduration=67.256482735 podStartE2EDuration="1m7.256482735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:17:55.240457783 +0000 UTC m=+67.913716817" watchObservedRunningTime="2026-07-16 22:17:55.256482735 +0000 UTC m=+67.929741745"
Jul 16 22:17:56.720982 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:56.720942    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:56.781250 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:56.781213    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:58.619510 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:58.619475    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:58.619879 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:58.619557    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:17:58.619879 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:58.619870    2556 scope.go:117] "RemoveContainer" containerID="555329c19ef6d86c0f7dd459d276ad1399b79de40dfe34cc86aec8d2aebfe33d"
Jul 16 22:17:58.620042 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:17:58.620020    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-4b8xc_openshift-console-operator(f36db1a6-eff5-45da-84a2-ca5d0572ac16)\"" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" podUID="f36db1a6-eff5-45da-84a2-ca5d0572ac16"
Jul 16 22:17:58.755053 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:58.755012    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:58.839150 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:58.839004    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:59.840687 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.840655    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"]
Jul 16 22:17:59.845430 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.843819    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"]
Jul 16 22:17:59.845430 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.844809    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:17:59.846165 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.846143    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"]
Jul 16 22:17:59.846286 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.846230    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:17:59.847765 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.847749    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 16 22:17:59.848329 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.848310    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 16 22:17:59.848437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.848393    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 16 22:17:59.848627 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.848612    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 16 22:17:59.848772 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.848757    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 16 22:17:59.849068 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.848875    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9frdq\""
Jul 16 22:17:59.849068 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.849032    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:17:59.852304 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.852285    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 16 22:17:59.852409 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.852356    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 16 22:17:59.852409 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.852367    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 16 22:17:59.852628 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.852612    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 16 22:17:59.862947 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.862925    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"]
Jul 16 22:17:59.873311 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.873273    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"]
Jul 16 22:17:59.874602 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.874578    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"]
Jul 16 22:17:59.941016 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.940980    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlcc8\" (UniqueName: \"kubernetes.io/projected/d4f654f2-dbc6-49a8-a15a-7eeeab1385fe-kube-api-access-vlcc8\") pod \"managed-serviceaccount-addon-agent-fdcd644f4-qr5lg\" (UID: \"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:17:59.941230 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5014d14a-d5b3-4f3d-a23f-e93528b61311-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:17:59.941230 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:17:59.941230 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941102    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r952g\" (UniqueName: \"kubernetes.io/projected/5014d14a-d5b3-4f3d-a23f-e93528b61311-kube-api-access-r952g\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:17:59.941413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941234    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ba0da3ab-646d-4c5a-9ced-60b2f263a953-klusterlet-config\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:17:59.941413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941312    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-hub\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:17:59.941413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941341    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-ca\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:17:59.941413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941369    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:17:59.941566 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941422    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch5cz\" (UniqueName: \"kubernetes.io/projected/ba0da3ab-646d-4c5a-9ced-60b2f263a953-kube-api-access-ch5cz\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:17:59.941566 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4f654f2-dbc6-49a8-a15a-7eeeab1385fe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fdcd644f4-qr5lg\" (UID: \"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:17:59.941566 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:17:59.941492    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba0da3ab-646d-4c5a-9ced-60b2f263a953-tmp\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.029002 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.028962    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:00.041936 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.041897    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5014d14a-d5b3-4f3d-a23f-e93528b61311-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.042251 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.041942    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.042251 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042089    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r952g\" (UniqueName: \"kubernetes.io/projected/5014d14a-d5b3-4f3d-a23f-e93528b61311-kube-api-access-r952g\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.042251 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042206    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ba0da3ab-646d-4c5a-9ced-60b2f263a953-klusterlet-config\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.042493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042287    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-hub\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.042493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042315    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-ca\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.042493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.042493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042389    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch5cz\" (UniqueName: \"kubernetes.io/projected/ba0da3ab-646d-4c5a-9ced-60b2f263a953-kube-api-access-ch5cz\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.042493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042443    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4f654f2-dbc6-49a8-a15a-7eeeab1385fe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fdcd644f4-qr5lg\" (UID: \"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:18:00.042493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042475    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba0da3ab-646d-4c5a-9ced-60b2f263a953-tmp\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.042797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042536    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlcc8\" (UniqueName: \"kubernetes.io/projected/d4f654f2-dbc6-49a8-a15a-7eeeab1385fe-kube-api-access-vlcc8\") pod \"managed-serviceaccount-addon-agent-fdcd644f4-qr5lg\" (UID: \"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:18:00.042873 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.042844    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5014d14a-d5b3-4f3d-a23f-e93528b61311-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.043330 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.043308    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba0da3ab-646d-4c5a-9ced-60b2f263a953-tmp\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.045569 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.045544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ba0da3ab-646d-4c5a-9ced-60b2f263a953-klusterlet-config\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.045678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.045591    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-ca\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.045744 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.045719    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.045852 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.045825    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d4f654f2-dbc6-49a8-a15a-7eeeab1385fe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fdcd644f4-qr5lg\" (UID: \"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:18:00.047220 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.047199    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.047303 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.047225    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5014d14a-d5b3-4f3d-a23f-e93528b61311-hub\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.056942 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.056914    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r952g\" (UniqueName: \"kubernetes.io/projected/5014d14a-d5b3-4f3d-a23f-e93528b61311-kube-api-access-r952g\") pod \"cluster-proxy-proxy-agent-67787b764-mtprv\" (UID: \"5014d14a-d5b3-4f3d-a23f-e93528b61311\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.059478 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.059452    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch5cz\" (UniqueName: \"kubernetes.io/projected/ba0da3ab-646d-4c5a-9ced-60b2f263a953-kube-api-access-ch5cz\") pod \"klusterlet-addon-workmgr-79b8769c9b-9xhcv\" (UID: \"ba0da3ab-646d-4c5a-9ced-60b2f263a953\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.060825 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.060801    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlcc8\" (UniqueName: \"kubernetes.io/projected/d4f654f2-dbc6-49a8-a15a-7eeeab1385fe-kube-api-access-vlcc8\") pod \"managed-serviceaccount-addon-agent-fdcd644f4-qr5lg\" (UID: \"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:18:00.125705 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.125616    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:00.143496 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.143458    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:18:00.146289 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.146262    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0db55b6f-7977-4d46-9528-a8dcf74de6cd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-kfn8p\" (UID: \"0db55b6f-7977-4d46-9528-a8dcf74de6cd\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:18:00.164694 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.164663    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"
Jul 16 22:18:00.172778 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.172751    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:00.178944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.178923    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"
Jul 16 22:18:00.224236 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.224202    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:00.324731 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.324691    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:00.344860 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.344815    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:00.345026 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.344914    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:00.345566 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.345543    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-service-ca-bundle\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:00.346917 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.346897    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7wd4s\""
Jul 16 22:18:00.347658 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.347636    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072-metrics-certs\") pod \"router-default-85f9cf987d-cs7ph\" (UID: \"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072\") " pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:00.355084 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.355065    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"
Jul 16 22:18:00.558206 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.558174    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c79bq\""
Jul 16 22:18:00.566069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:00.566042    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:01.519725 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:01.519655    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg"]
Jul 16 22:18:01.538051 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:01.538020    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p"]
Jul 16 22:18:01.542033 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:18:01.541991    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db55b6f_7977_4d46_9528_a8dcf74de6cd.slice/crio-0ad2b023362f80fe64bb60914a56b0b9fcf277eecc25c5d62aaa618b3d3d59c3 WatchSource:0}: Error finding container 0ad2b023362f80fe64bb60914a56b0b9fcf277eecc25c5d62aaa618b3d3d59c3: Status 404 returned error can't find the container with id 0ad2b023362f80fe64bb60914a56b0b9fcf277eecc25c5d62aaa618b3d3d59c3
Jul 16 22:18:01.737060 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:01.736973    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"]
Jul 16 22:18:01.739889 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:18:01.739860    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba0da3ab_646d_4c5a_9ced_60b2f263a953.slice/crio-97e5be05608264c0345d1793f4338cc51e3ac05c66dc14330fe841635fd7fed0 WatchSource:0}: Error finding container 97e5be05608264c0345d1793f4338cc51e3ac05c66dc14330fe841635fd7fed0: Status 404 returned error can't find the container with id 97e5be05608264c0345d1793f4338cc51e3ac05c66dc14330fe841635fd7fed0
Jul 16 22:18:01.785540 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:01.785517    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85f9cf987d-cs7ph"]
Jul 16 22:18:01.787792 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:01.787766    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv"]
Jul 16 22:18:01.788232 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:18:01.788201    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6e9d8e_e10f_4a60_b5d6_1c449b2a5072.slice/crio-3365ccd883f8733cc7563c52379336b867a6afeca0648ddb9ddde968fd92fff0 WatchSource:0}: Error finding container 3365ccd883f8733cc7563c52379336b867a6afeca0648ddb9ddde968fd92fff0: Status 404 returned error can't find the container with id 3365ccd883f8733cc7563c52379336b867a6afeca0648ddb9ddde968fd92fff0
Jul 16 22:18:01.790613 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:18:01.790586    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5014d14a_d5b3_4f3d_a23f_e93528b61311.slice/crio-a37d9814ac47884a28de1c89219997df5c071c92de5aa5cded2c22f92a5d969d WatchSource:0}: Error finding container a37d9814ac47884a28de1c89219997df5c071c92de5aa5cded2c22f92a5d969d: Status 404 returned error can't find the container with id a37d9814ac47884a28de1c89219997df5c071c92de5aa5cded2c22f92a5d969d
Jul 16 22:18:02.267561 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.267489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv" event={"ID":"ba0da3ab-646d-4c5a-9ced-60b2f263a953","Type":"ContainerStarted","Data":"97e5be05608264c0345d1793f4338cc51e3ac05c66dc14330fe841635fd7fed0"}
Jul 16 22:18:02.269132 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.269041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg" event={"ID":"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe","Type":"ContainerStarted","Data":"a84ce1be12f936339182baa6d4a35aa3c8dfb423ccdc6b74402476bc58d69cfc"}
Jul 16 22:18:02.270892 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.270861    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4qs4" event={"ID":"be6d1ea1-7566-4a06-bca8-47d0a98525de","Type":"ContainerStarted","Data":"ec3d1b91f46ad0a4f1f1db7b2847bd2c9c2ae21de83921f34764f3d3f8447039"}
Jul 16 22:18:02.272370 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.272345    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv" event={"ID":"5014d14a-d5b3-4f3d-a23f-e93528b61311","Type":"ContainerStarted","Data":"a37d9814ac47884a28de1c89219997df5c071c92de5aa5cded2c22f92a5d969d"}
Jul 16 22:18:02.273778 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.273755    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p" event={"ID":"0db55b6f-7977-4d46-9528-a8dcf74de6cd","Type":"ContainerStarted","Data":"0ad2b023362f80fe64bb60914a56b0b9fcf277eecc25c5d62aaa618b3d3d59c3"}
Jul 16 22:18:02.276817 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.276731    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zxmt" event={"ID":"a704ee01-ae37-4011-bb07-0a1fd7ca87b9","Type":"ContainerStarted","Data":"0cff707eef9da67a9f8083653fa8c0d94206d053b3f0938148537a571ab4f7fc"}
Jul 16 22:18:02.276817 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.276776    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zxmt" event={"ID":"a704ee01-ae37-4011-bb07-0a1fd7ca87b9","Type":"ContainerStarted","Data":"d71111d7f519f9604127f82eec52524a675574ed44e8e0067d2dc2483e7edf62"}
Jul 16 22:18:02.277052 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.277027    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:18:02.279198 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.279175    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhz6k" event={"ID":"c4d9b940-f340-4329-aea0-6252d6d331d0","Type":"ContainerStarted","Data":"aceefd577391a471f7b6aba3581760d00dd85b17761dad0021519a378e49e528"}
Jul 16 22:18:02.281362 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.280882    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85f9cf987d-cs7ph" event={"ID":"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072","Type":"ContainerStarted","Data":"1f601f4c5bef12d1e2a1a15c3d3d65d1cb91f3e8d3f8f01ae27a3b2424af8633"}
Jul 16 22:18:02.281362 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.280921    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85f9cf987d-cs7ph" event={"ID":"ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072","Type":"ContainerStarted","Data":"3365ccd883f8733cc7563c52379336b867a6afeca0648ddb9ddde968fd92fff0"}
Jul 16 22:18:02.306870 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.306819    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2zxmt" podStartSLOduration=34.006263364 podStartE2EDuration="40.306803613s" totalImagesPullingTime="6.300540249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:02.305470527 +0000 UTC m=+74.978729559" watchObservedRunningTime="2026-07-16 22:18:02.306803613 +0000 UTC m=+74.980062620"
Jul 16 22:18:02.370701 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.370511    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-85f9cf987d-cs7ph" podStartSLOduration=34.370488748 podStartE2EDuration="34.370488748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:02.370103751 +0000 UTC m=+75.043362760" watchObservedRunningTime="2026-07-16 22:18:02.370488748 +0000 UTC m=+75.043747757"
Jul 16 22:18:02.371359 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.371309    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hhz6k" podStartSLOduration=66.055487984 podStartE2EDuration="1m14.371297771s" totalImagesPullingTime="8.315809787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:02.331130205 +0000 UTC m=+75.004389227" watchObservedRunningTime="2026-07-16 22:18:02.371297771 +0000 UTC m=+75.044556778"
Jul 16 22:18:02.567359 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.567280    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:02.570436 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:02.570231    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:03.076629 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:03.076588    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:18:03.097626 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:03.097563    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3eec9f2-9166-4cdd-8a4c-01b227fddc76-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-m8xj9\" (UID: \"b3eec9f2-9166-4cdd-8a4c-01b227fddc76\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:18:03.199717 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:03.199494    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gt7tm\""
Jul 16 22:18:03.207808 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:03.207512    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"
Jul 16 22:18:03.291486 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:03.291040    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:03.292498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:03.292298    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-85f9cf987d-cs7ph"
Jul 16 22:18:03.442199 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:03.441896    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9"]
Jul 16 22:18:03.449632 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:18:03.449596    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3eec9f2_9166_4cdd_8a4c_01b227fddc76.slice/crio-d6197643f13308a5964494bbd6233f5a7a2a6760679bfddab8daa637dcebcd14 WatchSource:0}: Error finding container d6197643f13308a5964494bbd6233f5a7a2a6760679bfddab8daa637dcebcd14: Status 404 returned error can't find the container with id d6197643f13308a5964494bbd6233f5a7a2a6760679bfddab8daa637dcebcd14
Jul 16 22:18:04.299253 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:04.299174    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9" event={"ID":"b3eec9f2-9166-4cdd-8a4c-01b227fddc76","Type":"ContainerStarted","Data":"d6197643f13308a5964494bbd6233f5a7a2a6760679bfddab8daa637dcebcd14"}
Jul 16 22:18:04.598016 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:04.597923    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:04.661806 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:04.661760    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:05.122693 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:05.122652    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:05.186354 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:05.186311    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:05.241700 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:05.241660    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:05.672609 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:05.672572    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:05.746382 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:05.746333    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:06.074001 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:06.073957    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:06.138230 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:06.138198    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:11.726214 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:11.726170    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:11.806138 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:11.806086    2556 scope.go:117] "RemoveContainer" containerID="555329c19ef6d86c0f7dd459d276ad1399b79de40dfe34cc86aec8d2aebfe33d"
Jul 16 22:18:11.826635 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:11.826602    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:12.293410 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.293348    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2zxmt"
Jul 16 22:18:12.340189 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.339853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg" event={"ID":"d4f654f2-dbc6-49a8-a15a-7eeeab1385fe","Type":"ContainerStarted","Data":"0e4a326564e54ce89711f8a7a01a5dd03d4290b5da45ed2e67ed6c0f3dd8bc54"}
Jul 16 22:18:12.342248 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.342226    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:18:12.342384 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.342307    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" event={"ID":"f36db1a6-eff5-45da-84a2-ca5d0572ac16","Type":"ContainerStarted","Data":"c5a69d1646d81b1531aa77e8b1e44152f69c4882214018caa215c3dac8cc0aa3"}
Jul 16 22:18:12.344274 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.343066    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:18:12.344494 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.344454    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4qs4" event={"ID":"be6d1ea1-7566-4a06-bca8-47d0a98525de","Type":"ContainerStarted","Data":"cfa3b7b2109b7ec106dd3d1b36a9d40e1e1fd2ab216219b42e9c97915ea1da1a"}
Jul 16 22:18:12.349152 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.346837    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv" event={"ID":"5014d14a-d5b3-4f3d-a23f-e93528b61311","Type":"ContainerStarted","Data":"5ac41d4418e6efc6863028d23b84e592b3390903bfb1b5b5b5c5ab2a232e5d67"}
Jul 16 22:18:12.349152 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.346866    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv" event={"ID":"5014d14a-d5b3-4f3d-a23f-e93528b61311","Type":"ContainerStarted","Data":"e933388777c4047bdc326676908d4e5ebb0718da7ad70fd02d05470368d1871b"}
Jul 16 22:18:12.349152 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.346882    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv" event={"ID":"5014d14a-d5b3-4f3d-a23f-e93528b61311","Type":"ContainerStarted","Data":"731b692fd73ce89200340e0a8774d07391805e179b6d7fe3604a86af0230f844"}
Jul 16 22:18:12.349152 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.348839    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p" event={"ID":"0db55b6f-7977-4d46-9528-a8dcf74de6cd","Type":"ContainerStarted","Data":"c6f15e501aeb96d9ae409579bbaa51512eb844e0fcf9bd8c7b5b298ff470c77a"}
Jul 16 22:18:12.351804 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.351660    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9" event={"ID":"b3eec9f2-9166-4cdd-8a4c-01b227fddc76","Type":"ContainerStarted","Data":"3950ac52f2d2e3fe92505580c790c7369c361a7036371e7f0509f10616a3aa36"}
Jul 16 22:18:12.354900 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.353532    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv" event={"ID":"ba0da3ab-646d-4c5a-9ced-60b2f263a953","Type":"ContainerStarted","Data":"2eb7450ec4904922551a8fab7a800ca2d52136c9d1706da38c045d721cee236a"}
Jul 16 22:18:12.354900 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.354226    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:12.357523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.355646    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdcd644f4-qr5lg" podStartSLOduration=2.863853884 podStartE2EDuration="13.355634449s" totalImagesPullingTime="10.491780565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:12.355527404 +0000 UTC m=+85.028786444" watchObservedRunningTime="2026-07-16 22:18:12.355634449 +0000 UTC m=+85.028893457"
Jul 16 22:18:12.357523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.355907    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv"
Jul 16 22:18:12.373166 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.373018    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-m8xj9" podStartSLOduration=32.80683916 podStartE2EDuration="41.37300142s" totalImagesPullingTime="8.56616226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:12.372521859 +0000 UTC m=+85.045780868" watchObservedRunningTime="2026-07-16 22:18:12.37300142 +0000 UTC m=+85.046260428"
Jul 16 22:18:12.398488 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.398441    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b8769c9b-9xhcv" podStartSLOduration=3.096705376 podStartE2EDuration="13.398415471s" totalImagesPullingTime="10.301710095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:12.398014811 +0000 UTC m=+85.071273819" watchObservedRunningTime="2026-07-16 22:18:12.398415471 +0000 UTC m=+85.071674458"
Jul 16 22:18:12.423360 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.423303    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-kfn8p" podStartSLOduration=33.946698022 podStartE2EDuration="44.423281104s" totalImagesPullingTime="10.476583082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:12.4227459 +0000 UTC m=+85.096004942" watchObservedRunningTime="2026-07-16 22:18:12.423281104 +0000 UTC m=+85.096540114"
Jul 16 22:18:12.448606 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.448551    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67787b764-mtprv" podStartSLOduration=3.192362663 podStartE2EDuration="13.448530914s" totalImagesPullingTime="10.256168251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:12.447484288 +0000 UTC m=+85.120743296" watchObservedRunningTime="2026-07-16 22:18:12.448530914 +0000 UTC m=+85.121789921"
Jul 16 22:18:12.471842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.468760    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc" podStartSLOduration=38.005029981 podStartE2EDuration="44.468741305s" totalImagesPullingTime="6.463711324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:12.46472771 +0000 UTC m=+85.137986721" watchObservedRunningTime="2026-07-16 22:18:12.468741305 +0000 UTC m=+85.142000315"
Jul 16 22:18:12.475821 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.475790    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:12.734274 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.734148    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:12.832289 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:12.832195    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.062267 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:13.062211    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.276277 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:13.276249    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-4b8xc"
Jul 16 22:18:13.465046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:13.464959    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.563448 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:13.563415    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.623848 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:13.623766    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.779517 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:13.779425    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:14.132322 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:14.132286    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:14.329370 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:14.329333    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:14.415576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:14.415499    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:14.532360 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:14.532311    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:14.633378 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:14.633346    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:14.805395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:14.805335    2556 patch_prober.go:28] interesting pod/image-registry-779c67dc6c-q59db container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 16 22:18:14.805844 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:14.805406    2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-779c67dc6c-q59db" podUID="5f81de92-62c5-4b55-833d-1d1653130da3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 22:18:15.069581 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:15.069478    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:15.170003 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:15.169972    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:15.230939 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:15.230902    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:15.284838 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:15.284791    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:15.386162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:15.386044    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:15.581185 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:15.581080    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.233418 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:16.233390    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:18:16.373428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:16.373391    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4qs4" event={"ID":"be6d1ea1-7566-4a06-bca8-47d0a98525de","Type":"ContainerStarted","Data":"b10a17ea6fdeeb2bf9493d6c0f1c293492e0a392cde9441296af837cddcc7593"}
Jul 16 22:18:16.392047 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:16.391995    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-l4qs4" podStartSLOduration=17.068340634 podStartE2EDuration="39.391978154s" totalImagesPullingTime="22.32363752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:17:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:16.390785794 +0000 UTC m=+89.064044816" watchObservedRunningTime="2026-07-16 22:18:16.391978154 +0000 UTC m=+89.065237161"
Jul 16 22:18:16.438989 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:16.438958    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.495147 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:16.495036    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.557098 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:16.557035    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.675739 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:18.675701    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.750429 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:18.750395    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.804422 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:18.804387    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:19.320306 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:19.320260    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:19.420424 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:19.420384    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:19.520905 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:19.520868    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:19.792071 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:19.792038    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:19.848854 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:19.848820    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:21.410803 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:21.410766    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:21.511290 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:21.511253    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:22.961406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:22.961368    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:24.215717 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:24.215681    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-52pn4"
Jul 16 22:18:25.833566 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:25.833530    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:25.945503 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:25.945470    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.047901 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.047854    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.127631 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.127596    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dlnzt"]
Jul 16 22:18:26.134918 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.134894    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.138608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.138584    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 22:18:26.138738 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.138639    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 22:18:26.138838 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.138812    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n649k\""
Jul 16 22:18:26.139149 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.139133    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 22:18:26.139835 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.139816    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 22:18:26.156799 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.156772    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.168794 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.168767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-tls\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.168910 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.168803    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.168910 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.168831    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-textfile\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.169027 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.168916    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd7152-05d7-44fd-9228-09ae575a2f28-metrics-client-ca\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.169027 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.168984    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-root\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.169161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.169044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-wtmp\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.169161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.169076    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8n2l\" (UniqueName: \"kubernetes.io/projected/4ecd7152-05d7-44fd-9228-09ae575a2f28-kube-api-access-d8n2l\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.169161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.169103    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-sys\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.169325 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.169153    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-accelerators-collector-config\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.228962 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.228918    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.269879 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.269845    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-accelerators-collector-config\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.269896    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-tls\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.269924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.269949    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-textfile\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.269976    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd7152-05d7-44fd-9228-09ae575a2f28-metrics-client-ca\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270259 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:18:26.270075    2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 16 22:18:26.270259 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-root\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270259 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:18:26.270196    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-tls podName:4ecd7152-05d7-44fd-9228-09ae575a2f28 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:26.770164862 +0000 UTC m=+99.443423867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-tls") pod "node-exporter-dlnzt" (UID: "4ecd7152-05d7-44fd-9228-09ae575a2f28") : secret "node-exporter-tls" not found
Jul 16 22:18:26.270259 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270222    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-root\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270259 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270229    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-wtmp\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270264    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8n2l\" (UniqueName: \"kubernetes.io/projected/4ecd7152-05d7-44fd-9228-09ae575a2f28-kube-api-access-d8n2l\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270294    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-sys\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270363    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-sys\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270410    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-textfile\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270484    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-wtmp\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270718 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270610    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-accelerators-collector-config\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.270718 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.270615    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ecd7152-05d7-44fd-9228-09ae575a2f28-metrics-client-ca\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.272551 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.272531    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.280194 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.280170    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8n2l\" (UniqueName: \"kubernetes.io/projected/4ecd7152-05d7-44fd-9228-09ae575a2f28-kube-api-access-d8n2l\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.328627 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.328591    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.384743 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.384658    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.453102 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.453069    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.516536 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.516496    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.739569 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.739474    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.775201 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.775169    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-tls\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.777756 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.777734    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ecd7152-05d7-44fd-9228-09ae575a2f28-node-exporter-tls\") pod \"node-exporter-dlnzt\" (UID: \"4ecd7152-05d7-44fd-9228-09ae575a2f28\") " pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:26.840382 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.840317    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.976370 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:26.976338    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-779c67dc6c-q59db"]
Jul 16 22:18:27.045738 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:27.045708    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dlnzt"
Jul 16 22:18:27.064966 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:18:27.064932    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecd7152_05d7_44fd_9228_09ae575a2f28.slice/crio-29d2f69d3a55c09cd58d42321c47510ced4bf3621a02241859e86086b18e4e0f WatchSource:0}: Error finding container 29d2f69d3a55c09cd58d42321c47510ced4bf3621a02241859e86086b18e4e0f: Status 404 returned error can't find the container with id 29d2f69d3a55c09cd58d42321c47510ced4bf3621a02241859e86086b18e4e0f
Jul 16 22:18:27.083442 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:27.083383    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:27.178639 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:27.178601    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:27.263526 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:27.263487    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:27.416421 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:27.416328    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dlnzt" event={"ID":"4ecd7152-05d7-44fd-9228-09ae575a2f28","Type":"ContainerStarted","Data":"29d2f69d3a55c09cd58d42321c47510ced4bf3621a02241859e86086b18e4e0f"}
Jul 16 22:18:28.760141 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:28.760088    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:28.860899 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:28.860788    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:29.086262 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:29.086229    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:29.151401 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:29.151317    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:29.267458 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:29.267424    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:29.426686 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:29.426585    2556 generic.go:356] "Generic (PLEG): container finished" podID="4ecd7152-05d7-44fd-9228-09ae575a2f28" containerID="e639c5e6d3fc0da04133ffe81644f924a61b35aa5d03bda055af847bf465cc47" exitCode=0
Jul 16 22:18:29.426842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:29.426675    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dlnzt" event={"ID":"4ecd7152-05d7-44fd-9228-09ae575a2f28","Type":"ContainerDied","Data":"e639c5e6d3fc0da04133ffe81644f924a61b35aa5d03bda055af847bf465cc47"}
Jul 16 22:18:29.629699 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:29.629663    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:29.928543 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:29.928496    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:30.215680 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:30.215580    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:30.433561 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:30.433526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dlnzt" event={"ID":"4ecd7152-05d7-44fd-9228-09ae575a2f28","Type":"ContainerStarted","Data":"d9b1d0b859a13db9e7a5c18bde7ba48ce85a3eecd519a00eacc9aa14c0797a91"}
Jul 16 22:18:30.433733 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:30.433577    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dlnzt" event={"ID":"4ecd7152-05d7-44fd-9228-09ae575a2f28","Type":"ContainerStarted","Data":"1ddba0a96b96e1ab9a3a95c1e5de20ef6aa23b17852716364b384ddf8eeed289"}
Jul 16 22:18:30.458776 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:30.458722    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dlnzt" podStartSLOduration=2.871432819 podStartE2EDuration="4.458707121s" totalImagesPullingTime="1.514628239s" totalInitContainerRuntime="72.646063ms" isStatefulPod=true podCreationTimestamp="2026-07-16 22:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:30.457783098 +0000 UTC m=+103.131042114" watchObservedRunningTime="2026-07-16 22:18:30.458707121 +0000 UTC m=+103.131966168"
Jul 16 22:18:30.841657 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:30.841621    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:30.942135 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:30.942082    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:31.179357 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:31.179277    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:32.299372 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:32.299329    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:32.374490 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:32.374456    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:32.568613 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:32.568531    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:32.669514 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:32.669485    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:33.086871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:33.086830    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:33.153365 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:33.153321    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:33.255253 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:33.255223    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:33.357383 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:33.357291    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:34.183692 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:34.183657    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:34.279945 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:34.279905    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:39.170797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:39.170757    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:39.245921 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:39.245880    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:41.477385 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:41.477245    2556 generic.go:356] "Generic (PLEG): container finished" podID="387137d2-c707-406b-a263-8df8fd8d1d35" containerID="c9a77e95b66876a230d801796ca0ea13bf9a72e7392bdf9b97cefb81e842f988" exitCode=0
Jul 16 22:18:41.477385 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:41.477299    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw" event={"ID":"387137d2-c707-406b-a263-8df8fd8d1d35","Type":"ContainerDied","Data":"c9a77e95b66876a230d801796ca0ea13bf9a72e7392bdf9b97cefb81e842f988"}
Jul 16 22:18:41.477894 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:41.477707    2556 scope.go:117] "RemoveContainer" containerID="c9a77e95b66876a230d801796ca0ea13bf9a72e7392bdf9b97cefb81e842f988"
Jul 16 22:18:41.741227 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:41.741140    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:41.842299 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:41.842269    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:42.086470 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:42.086428    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:42.483429 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:42.483350    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8f4fw" event={"ID":"387137d2-c707-406b-a263-8df8fd8d1d35","Type":"ContainerStarted","Data":"d53c643e28aacd499b9ac65fc395c6c53ec6b841bdded66ff4e5bdee9c03a9b8"}
Jul 16 22:18:42.484856 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:42.484834    2556 generic.go:356] "Generic (PLEG): container finished" podID="75f6164d-4c29-4111-bdaf-961f00f83f12" containerID="48699b41ed91f98c36c443d2ee5e2e995032bf9557425b7248b0722cc8862c4b" exitCode=0
Jul 16 22:18:42.484975 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:42.484884    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sf45x" event={"ID":"75f6164d-4c29-4111-bdaf-961f00f83f12","Type":"ContainerDied","Data":"48699b41ed91f98c36c443d2ee5e2e995032bf9557425b7248b0722cc8862c4b"}
Jul 16 22:18:42.485183 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:42.485169    2556 scope.go:117] "RemoveContainer" containerID="48699b41ed91f98c36c443d2ee5e2e995032bf9557425b7248b0722cc8862c4b"
Jul 16 22:18:43.490933 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:43.490899    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sf45x" event={"ID":"75f6164d-4c29-4111-bdaf-961f00f83f12","Type":"ContainerStarted","Data":"56b680222e45d40cd533cce684069c8f202c233556f3488eee2569293089ee27"}
Jul 16 22:18:45.293590 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:45.293553    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:45.395535 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:45.395500    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:45.491856 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:45.491818    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:49.503709 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:49.503666    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:49.590792 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:49.590755    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:51.394570 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:51.394519    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:51.492748 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:51.492711    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:51.592390 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:51.592358    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:51.669660 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:51.669572    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:51.997163 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:51.997066    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-779c67dc6c-q59db" podUID="5f81de92-62c5-4b55-833d-1d1653130da3" containerName="registry" containerID="cri-o://58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82" gracePeriod=30
Jul 16 22:18:52.245903 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.245881    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:18:52.398430 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398403    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-trusted-ca\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398445    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-registry-certificates\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398475    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m28kz\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-kube-api-access-m28kz\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398654    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-installation-pull-secrets\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398688    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-image-registry-private-configuration\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398767    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398794    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-bound-sa-token\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398845    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f81de92-62c5-4b55-833d-1d1653130da3-ca-trust-extracted\") pod \"5f81de92-62c5-4b55-833d-1d1653130da3\" (UID: \"5f81de92-62c5-4b55-833d-1d1653130da3\") "
Jul 16 22:18:52.398880 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398861    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:18:52.399293 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.398881    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:18:52.399293 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.399049    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-trusted-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.399293 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.399061    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f81de92-62c5-4b55-833d-1d1653130da3-registry-certificates\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.401163 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.401097    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:18:52.401290 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.401226    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-kube-api-access-m28kz" (OuterVolumeSpecName: "kube-api-access-m28kz") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "kube-api-access-m28kz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:18:52.401629 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.401605    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:18:52.401720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.401683    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:18:52.401770 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.401713    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:18:52.407692 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.407669    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f81de92-62c5-4b55-833d-1d1653130da3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5f81de92-62c5-4b55-833d-1d1653130da3" (UID: "5f81de92-62c5-4b55-833d-1d1653130da3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:18:52.499789 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.499758    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-registry-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.499789 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.499784    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-bound-sa-token\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.499789 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.499793    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f81de92-62c5-4b55-833d-1d1653130da3-ca-trust-extracted\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.499964 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.499802    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m28kz\" (UniqueName: \"kubernetes.io/projected/5f81de92-62c5-4b55-833d-1d1653130da3-kube-api-access-m28kz\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.499964 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.499811    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-installation-pull-secrets\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.499964 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.499823    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f81de92-62c5-4b55-833d-1d1653130da3-image-registry-private-configuration\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:18:52.525001 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.524974    2556 generic.go:356] "Generic (PLEG): container finished" podID="5f81de92-62c5-4b55-833d-1d1653130da3" containerID="58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82" exitCode=0
Jul 16 22:18:52.525137 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.525028    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779c67dc6c-q59db"
Jul 16 22:18:52.525137 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.525060    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779c67dc6c-q59db" event={"ID":"5f81de92-62c5-4b55-833d-1d1653130da3","Type":"ContainerDied","Data":"58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82"}
Jul 16 22:18:52.525137 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.525126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779c67dc6c-q59db" event={"ID":"5f81de92-62c5-4b55-833d-1d1653130da3","Type":"ContainerDied","Data":"c4073f4b470f4d46d94fa416927571fa48112877fc6f081522de9be708469069"}
Jul 16 22:18:52.525287 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.525148    2556 scope.go:117] "RemoveContainer" containerID="58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82"
Jul 16 22:18:52.533884 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.533864    2556 scope.go:117] "RemoveContainer" containerID="58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82"
Jul 16 22:18:52.534177 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:18:52.534151    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82\": container with ID starting with 58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82 not found: ID does not exist" containerID="58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82"
Jul 16 22:18:52.534231 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.534191    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82"} err="failed to get container status \"58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82\": rpc error: code = NotFound desc = could not find container \"58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82\": container with ID starting with 58bef13cb5128931b827d3da2a0f02fe8389421e03819df0ad105f5e6546ff82 not found: ID does not exist"
Jul 16 22:18:52.546324 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.546305    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-779c67dc6c-q59db"]
Jul 16 22:18:52.549551 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:52.549530    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-779c67dc6c-q59db"]
Jul 16 22:18:53.812141 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:53.811684    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f81de92-62c5-4b55-833d-1d1653130da3" path="/var/lib/kubelet/pods/5f81de92-62c5-4b55-833d-1d1653130da3/volumes"
Jul 16 22:18:56.542408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:56.542380    2556 generic.go:356] "Generic (PLEG): container finished" podID="4a08215e-447d-4500-8136-3a196037bc48" containerID="c0030309202becade18743282b2b7d1ec2516abf63e02dae19265c5cd8677cec" exitCode=0
Jul 16 22:18:56.542764 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:56.542457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f" event={"ID":"4a08215e-447d-4500-8136-3a196037bc48","Type":"ContainerDied","Data":"c0030309202becade18743282b2b7d1ec2516abf63e02dae19265c5cd8677cec"}
Jul 16 22:18:56.542804 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:56.542769    2556 scope.go:117] "RemoveContainer" containerID="c0030309202becade18743282b2b7d1ec2516abf63e02dae19265c5cd8677cec"
Jul 16 22:18:56.770737 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:56.770696    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:56.871615 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:56.871539    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:57.547395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:18:57.547357    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7c55f" event={"ID":"4a08215e-447d-4500-8136-3a196037bc48","Type":"ContainerStarted","Data":"78da0f92b5ccd4a08184b72dd0889981e97a38a13f20bb39a1a0397b316fa7a6"}
Jul 16 22:19:11.790304 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:11.790266    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:11.864093 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:11.864051    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:42.203752 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:42.203672    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:42.287995 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:42.287955    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:51.960911 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:51.960879    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:52.065784 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:52.065748    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:54.182907 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:54.182867    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:54.249446 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:54.249407    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:56.740018 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:56.739985    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:56.839164 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:19:56.839124    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:02.599419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:02.599380    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:02.661101 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:02.661061    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:02.727177 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:02.727133    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:19.190604 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:19.190567    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:19.288954 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:19.288916    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:33.236360 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:33.236316    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:46.223389 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:46.223346    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:47.610401 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:47.610371    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:49.872308 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:49.872275    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:49.977773 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:49.977730    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:51.032442 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:51.032405    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:51.144046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:51.144011    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:52.250512 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:52.250478    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:52.345987 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:52.345953    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:54.165170 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:54.165139    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:54.574620 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:54.574587    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:56.840575 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:56.840508    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:56.903029 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:20:56.902989    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:02.808957 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:02.808921    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:06.472402 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:06.472367    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:06.576897 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:06.576850    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:12.142394 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:12.142314    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:12.248974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:12.248939    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:14.292696 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:14.292659    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:24.728752 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:24.728720    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:24.846436 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:24.846403    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:26.399831 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:26.399743    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:26.499557 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:26.499524    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:28.236475 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:28.236430    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:28.332546 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:28.332513    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:28.898014 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:28.897971    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:29.193601 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.193512    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:29.300496 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.300434    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:29.340419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.340387    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"]
Jul 16 22:21:29.340683 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.340671    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f81de92-62c5-4b55-833d-1d1653130da3" containerName="registry"
Jul 16 22:21:29.340738 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.340686    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f81de92-62c5-4b55-833d-1d1653130da3" containerName="registry"
Jul 16 22:21:29.340772 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.340752    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f81de92-62c5-4b55-833d-1d1653130da3" containerName="registry"
Jul 16 22:21:29.342840 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.342824    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.345524 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.345504    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 22:21:29.346067 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.346048    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 22:21:29.346178 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.346066    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 22:21:29.346242 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.346207    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 22:21:29.346282 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.346248    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-592w7\""
Jul 16 22:21:29.375266 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.375232    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"]
Jul 16 22:21:29.435229 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.435184    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f62ec0fd-06de-4573-9942-2cc77b530d9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.435229 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.435231    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f62ec0fd-06de-4573-9942-2cc77b530d9a-tmp\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.435463 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.435306    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f62ec0fd-06de-4573-9942-2cc77b530d9a-manifests\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.435463 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.435371    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54shk\" (UniqueName: \"kubernetes.io/projected/f62ec0fd-06de-4573-9942-2cc77b530d9a-kube-api-access-54shk\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.435463 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.435414    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f62ec0fd-06de-4573-9942-2cc77b530d9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.472785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.472700    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:29.536628 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.536592    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f62ec0fd-06de-4573-9942-2cc77b530d9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.536628 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.536628    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f62ec0fd-06de-4573-9942-2cc77b530d9a-tmp\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.536874 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.536663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f62ec0fd-06de-4573-9942-2cc77b530d9a-manifests\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.536874 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.536783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54shk\" (UniqueName: \"kubernetes.io/projected/f62ec0fd-06de-4573-9942-2cc77b530d9a-kube-api-access-54shk\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.536874 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.536841    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f62ec0fd-06de-4573-9942-2cc77b530d9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.537033 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.536989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f62ec0fd-06de-4573-9942-2cc77b530d9a-manifests\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.539248 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.539220    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f62ec0fd-06de-4573-9942-2cc77b530d9a-tmp\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.539449 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.539429    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f62ec0fd-06de-4573-9942-2cc77b530d9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.539490 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.539430    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f62ec0fd-06de-4573-9942-2cc77b530d9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.549684 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.549661    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54shk\" (UniqueName: \"kubernetes.io/projected/f62ec0fd-06de-4573-9942-2cc77b530d9a-kube-api-access-54shk\") pod \"opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8\" (UID: \"f62ec0fd-06de-4573-9942-2cc77b530d9a\") " pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.571187 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.571150    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:29.654416 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.654382    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:29.672525 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.672496    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:29.828665 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:21:29.828634    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62ec0fd_06de_4573_9942_2cc77b530d9a.slice/crio-248fedd0554f93e597e2a3c20158ddee7a7cfe1a6c970ff4706cd1c68b84ac96 WatchSource:0}: Error finding container 248fedd0554f93e597e2a3c20158ddee7a7cfe1a6c970ff4706cd1c68b84ac96: Status 404 returned error can't find the container with id 248fedd0554f93e597e2a3c20158ddee7a7cfe1a6c970ff4706cd1c68b84ac96
Jul 16 22:21:29.829167 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.829147    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"]
Jul 16 22:21:29.945303 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:29.945265    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:30.045785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:30.045741    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:30.123400 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:30.123367    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8" event={"ID":"f62ec0fd-06de-4573-9942-2cc77b530d9a","Type":"ContainerStarted","Data":"248fedd0554f93e597e2a3c20158ddee7a7cfe1a6c970ff4706cd1c68b84ac96"}
Jul 16 22:21:30.691403 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:30.691364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:30.791019 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:30.790972    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:33.140949 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:33.140907    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8" event={"ID":"f62ec0fd-06de-4573-9942-2cc77b530d9a","Type":"ContainerStarted","Data":"264f71057affab2bea6e2c6474511ddfe9d7e08a0af8ef44521e3a86813b4b90"}
Jul 16 22:21:34.147441 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:34.147408    2556 generic.go:356] "Generic (PLEG): container finished" podID="f62ec0fd-06de-4573-9942-2cc77b530d9a" containerID="264f71057affab2bea6e2c6474511ddfe9d7e08a0af8ef44521e3a86813b4b90" exitCode=0
Jul 16 22:21:34.147838 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:34.147492    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8" event={"ID":"f62ec0fd-06de-4573-9942-2cc77b530d9a","Type":"ContainerDied","Data":"264f71057affab2bea6e2c6474511ddfe9d7e08a0af8ef44521e3a86813b4b90"}
Jul 16 22:21:35.155615 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:35.155568    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8" event={"ID":"f62ec0fd-06de-4573-9942-2cc77b530d9a","Type":"ContainerStarted","Data":"7d2962adda6dcb6fc764842fb1749cb6a97651f67ff8419d17d0b238c2e075c4"}
Jul 16 22:21:35.156100 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:35.155708    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:35.178298 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:35.178246    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8" podStartSLOduration=1.564749664 podStartE2EDuration="6.178228269s" totalImagesPullingTime="3.573375634s" totalInitContainerRuntime="1.040102971s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:21:35.176766904 +0000 UTC m=+287.850025917" watchObservedRunningTime="2026-07-16 22:21:35.178228269 +0000 UTC m=+287.851487276"
Jul 16 22:21:40.627677 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:40.627640    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:40.737536 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:40.737494    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:40.846211 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:40.846176    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:40.951426 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:40.951327    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:46.162338 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:46.162305    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8"
Jul 16 22:21:47.709891 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:47.709860    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:21:47.713685 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:47.713662    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:21:47.714826 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:47.714808    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:21:47.719180 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:47.719157    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:21:47.722748 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:47.722726    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 22:21:48.497811 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:48.497747    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:48.709861 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:48.709828    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:48.970785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:48.970748    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:49.136591 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.136549    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"]
Jul 16 22:21:49.139488 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.139458    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.142562 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.142535    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 22:21:49.142832 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.142813    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cnqmk\""
Jul 16 22:21:49.143252 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.143232    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 22:21:49.159162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.159132    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"]
Jul 16 22:21:49.201100 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.201066    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn2bt\" (UniqueName: \"kubernetes.io/projected/41d05597-6cf9-448f-8870-cac62db55f62-kube-api-access-qn2bt\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.201292 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.201126    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41d05597-6cf9-448f-8870-cac62db55f62-tmp\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.201292 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.201158    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d05597-6cf9-448f-8870-cac62db55f62-tls-certs\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.302455 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.302418    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn2bt\" (UniqueName: \"kubernetes.io/projected/41d05597-6cf9-448f-8870-cac62db55f62-kube-api-access-qn2bt\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.302455 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.302457    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41d05597-6cf9-448f-8870-cac62db55f62-tmp\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.302720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.302621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d05597-6cf9-448f-8870-cac62db55f62-tls-certs\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.304992 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.304965    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41d05597-6cf9-448f-8870-cac62db55f62-tmp\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.305868 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.305846    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d05597-6cf9-448f-8870-cac62db55f62-tls-certs\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.329032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.328989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn2bt\" (UniqueName: \"kubernetes.io/projected/41d05597-6cf9-448f-8870-cac62db55f62-kube-api-access-qn2bt\") pod \"kube-auth-proxy-7b969c6b8c-fjvw6\" (UID: \"41d05597-6cf9-448f-8870-cac62db55f62\") " pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.386327 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.386283    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ng8d4"]
Jul 16 22:21:49.388734 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.388701    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:49.392588 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.392572    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cgqsx\""
Jul 16 22:21:49.396200 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.396182    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 16 22:21:49.413489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.413458    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ng8d4"]
Jul 16 22:21:49.445229 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.445188    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:49.449957 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.449936    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"
Jul 16 22:21:49.506380 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.505028    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xjj4\" (UniqueName: \"kubernetes.io/projected/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-kube-api-access-7xjj4\") pod \"odh-model-controller-7998c67dc4-ng8d4\" (UID: \"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:49.506380 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.505101    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-cert\") pod \"odh-model-controller-7998c67dc4-ng8d4\" (UID: \"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:49.598550 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.598468    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6"]
Jul 16 22:21:49.601469 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:21:49.601442    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d05597_6cf9_448f_8870_cac62db55f62.slice/crio-5261a7ffc8dce55a18a64c44b0dcbcfa3c5292fde03abd97f947aa8fe0694781 WatchSource:0}: Error finding container 5261a7ffc8dce55a18a64c44b0dcbcfa3c5292fde03abd97f947aa8fe0694781: Status 404 returned error can't find the container with id 5261a7ffc8dce55a18a64c44b0dcbcfa3c5292fde03abd97f947aa8fe0694781
Jul 16 22:21:49.603312 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.603293    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:21:49.606161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.606136    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xjj4\" (UniqueName: \"kubernetes.io/projected/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-kube-api-access-7xjj4\") pod \"odh-model-controller-7998c67dc4-ng8d4\" (UID: \"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:49.606245 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.606206    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-cert\") pod \"odh-model-controller-7998c67dc4-ng8d4\" (UID: \"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:49.606416 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:21:49.606394    2556 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 16 22:21:49.606485 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:21:49.606480    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-cert podName:eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3 nodeName:}" failed. No retries permitted until 2026-07-16 22:21:50.10645854 +0000 UTC m=+302.779717535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-cert") pod "odh-model-controller-7998c67dc4-ng8d4" (UID: "eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3") : secret "odh-model-controller-webhook-cert" not found
Jul 16 22:21:49.619528 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:49.619504    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xjj4\" (UniqueName: \"kubernetes.io/projected/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-kube-api-access-7xjj4\") pod \"odh-model-controller-7998c67dc4-ng8d4\" (UID: \"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:50.111548 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.111507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-cert\") pod \"odh-model-controller-7998c67dc4-ng8d4\" (UID: \"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:50.114012 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.113992    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3-cert\") pod \"odh-model-controller-7998c67dc4-ng8d4\" (UID: \"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:50.220579 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.220540    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6" event={"ID":"41d05597-6cf9-448f-8870-cac62db55f62","Type":"ContainerStarted","Data":"5261a7ffc8dce55a18a64c44b0dcbcfa3c5292fde03abd97f947aa8fe0694781"}
Jul 16 22:21:50.298904 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.298865    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:50.357068 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.357012    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nlf8k"]
Jul 16 22:21:50.359708 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.359640    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:50.366836 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.365165    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 16 22:21:50.378924 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.378850    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nlf8k"]
Jul 16 22:21:50.414822 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.414648    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn4wj\" (UniqueName: \"kubernetes.io/projected/80bc537d-de13-46fa-bc76-15f1f096157b-kube-api-access-cn4wj\") pod \"model-serving-api-54d89bd5fc-nlf8k\" (UID: \"80bc537d-de13-46fa-bc76-15f1f096157b\") " pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:50.414822 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.414729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80bc537d-de13-46fa-bc76-15f1f096157b-tls-certs\") pod \"model-serving-api-54d89bd5fc-nlf8k\" (UID: \"80bc537d-de13-46fa-bc76-15f1f096157b\") " pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:50.475247 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.475168    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ng8d4"]
Jul 16 22:21:50.516161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.516100    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80bc537d-de13-46fa-bc76-15f1f096157b-tls-certs\") pod \"model-serving-api-54d89bd5fc-nlf8k\" (UID: \"80bc537d-de13-46fa-bc76-15f1f096157b\") " pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:50.516318 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.516224    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn4wj\" (UniqueName: \"kubernetes.io/projected/80bc537d-de13-46fa-bc76-15f1f096157b-kube-api-access-cn4wj\") pod \"model-serving-api-54d89bd5fc-nlf8k\" (UID: \"80bc537d-de13-46fa-bc76-15f1f096157b\") " pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:50.516318 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:21:50.516269    2556 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 16 22:21:50.516427 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:21:50.516345    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80bc537d-de13-46fa-bc76-15f1f096157b-tls-certs podName:80bc537d-de13-46fa-bc76-15f1f096157b nodeName:}" failed. No retries permitted until 2026-07-16 22:21:51.016322665 +0000 UTC m=+303.689581652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/80bc537d-de13-46fa-bc76-15f1f096157b-tls-certs") pod "model-serving-api-54d89bd5fc-nlf8k" (UID: "80bc537d-de13-46fa-bc76-15f1f096157b") : secret "model-serving-api-tls" not found
Jul 16 22:21:50.539360 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.539326    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn4wj\" (UniqueName: \"kubernetes.io/projected/80bc537d-de13-46fa-bc76-15f1f096157b-kube-api-access-cn4wj\") pod \"model-serving-api-54d89bd5fc-nlf8k\" (UID: \"80bc537d-de13-46fa-bc76-15f1f096157b\") " pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:50.630256 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.630148    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"]
Jul 16 22:21:50.633127 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.633092    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.636944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.636767    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 22:21:50.636944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.636818    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ksph6\""
Jul 16 22:21:50.637159 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.637038    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 22:21:50.637159 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.637074    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 22:21:50.637292 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.637251    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 22:21:50.645196 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.645170    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"]
Jul 16 22:21:50.718466 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.718432    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-cacerts\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.718660 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.718502    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.718660 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.718571    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwrdz\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-kube-api-access-gwrdz\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.718660 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.718626    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-istio-token\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.718830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.718671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a52781-2011-4433-992e-fb32748f9bfd-local-certs\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.718830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.718696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.718830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.718718    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.819200 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.819160    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwrdz\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-kube-api-access-gwrdz\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.819386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.819216    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-istio-token\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.819386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.819266    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a52781-2011-4433-992e-fb32748f9bfd-local-certs\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.819386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.819288    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.819386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.819309    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.819386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.819351    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-cacerts\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.819607 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.819394    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.820406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.820349    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.822104 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.822076    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-cacerts\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.822401 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.822380    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a52781-2011-4433-992e-fb32748f9bfd-local-certs\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.822489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.822440    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.822489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.822480    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.827711 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.827683    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-istio-token\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.828468 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.828442    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwrdz\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-kube-api-access-gwrdz\") pod \"istiod-openshift-gateway-7574459656-lwf4s\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:50.927640 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.927542    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:50.949206 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:50.949164    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:51.021916 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.021882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80bc537d-de13-46fa-bc76-15f1f096157b-tls-certs\") pod \"model-serving-api-54d89bd5fc-nlf8k\" (UID: \"80bc537d-de13-46fa-bc76-15f1f096157b\") " pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:51.024375 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.024348    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80bc537d-de13-46fa-bc76-15f1f096157b-tls-certs\") pod \"model-serving-api-54d89bd5fc-nlf8k\" (UID: \"80bc537d-de13-46fa-bc76-15f1f096157b\") " pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:51.100417 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.100279    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"]
Jul 16 22:21:51.102633 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:21:51.102606    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a52781_2011_4433_992e_fb32748f9bfd.slice/crio-d092473676e5150005e589a88759c6823e1e35b63ba90bf97fe11a1de407edf6 WatchSource:0}: Error finding container d092473676e5150005e589a88759c6823e1e35b63ba90bf97fe11a1de407edf6: Status 404 returned error can't find the container with id d092473676e5150005e589a88759c6823e1e35b63ba90bf97fe11a1de407edf6
Jul 16 22:21:51.227314 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.227204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4" event={"ID":"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3","Type":"ContainerStarted","Data":"a41f55c9ed06bed57949cbdcd7530875584733f6f29ac1708680d5b26a204175"}
Jul 16 22:21:51.228893 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.228858    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s" event={"ID":"48a52781-2011-4433-992e-fb32748f9bfd","Type":"ContainerStarted","Data":"d092473676e5150005e589a88759c6823e1e35b63ba90bf97fe11a1de407edf6"}
Jul 16 22:21:51.278284 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.278249    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:51.515951 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.515468    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nlf8k"]
Jul 16 22:21:51.518948 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:21:51.518917    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80bc537d_de13_46fa_bc76_15f1f096157b.slice/crio-e172fa64590a4a2b08623315167fe3f22b8f22600c8858687ec163b957029f6a WatchSource:0}: Error finding container e172fa64590a4a2b08623315167fe3f22b8f22600c8858687ec163b957029f6a: Status 404 returned error can't find the container with id e172fa64590a4a2b08623315167fe3f22b8f22600c8858687ec163b957029f6a
Jul 16 22:21:51.755138 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:51.755079    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:52.236641 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:52.236583    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k" event={"ID":"80bc537d-de13-46fa-bc76-15f1f096157b","Type":"ContainerStarted","Data":"e172fa64590a4a2b08623315167fe3f22b8f22600c8858687ec163b957029f6a"}
Jul 16 22:21:55.256181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.256105    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4" event={"ID":"eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3","Type":"ContainerStarted","Data":"9cc1674836ac83e207c1cf75704f4da036e941b03dd12c24f347223ce8f11f87"}
Jul 16 22:21:55.256689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.256206    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:21:55.257834 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.257806    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k" event={"ID":"80bc537d-de13-46fa-bc76-15f1f096157b","Type":"ContainerStarted","Data":"81dbc8d7cf1ce297d55edad953632cc71662901c568d28e5e63fb2458dc61aef"}
Jul 16 22:21:55.257950 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.257884    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:21:55.259368 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.259341    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6" event={"ID":"41d05597-6cf9-448f-8870-cac62db55f62","Type":"ContainerStarted","Data":"0cceb13bd5bb448d374586885aa0c6bb35ac4b17c0ee5ec127a7e5d4c6ae3fcc"}
Jul 16 22:21:55.273121 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.273075    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4" podStartSLOduration=2.258617967 podStartE2EDuration="6.273063012s" totalImagesPullingTime="4.014445045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:21:55.271964502 +0000 UTC m=+307.945223525" watchObservedRunningTime="2026-07-16 22:21:55.273063012 +0000 UTC m=+307.946322021"
Jul 16 22:21:55.289515 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.289465    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b969c6b8c-fjvw6" podStartSLOduration=1.814927511 podStartE2EDuration="6.289448782s" totalImagesPullingTime="4.474521271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:21:55.287349963 +0000 UTC m=+307.960608962" watchObservedRunningTime="2026-07-16 22:21:55.289448782 +0000 UTC m=+307.962707790"
Jul 16 22:21:55.304048 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:55.304004    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k" podStartSLOduration=2.288082015 podStartE2EDuration="5.303991301s" totalImagesPullingTime="3.015909286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:21:55.302566384 +0000 UTC m=+307.975825430" watchObservedRunningTime="2026-07-16 22:21:55.303991301 +0000 UTC m=+307.977250308"
Jul 16 22:21:57.101653 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:57.101607    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 16 22:21:57.101900 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:57.101690    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 16 22:21:57.271121 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:57.271078    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s" event={"ID":"48a52781-2011-4433-992e-fb32748f9bfd","Type":"ContainerStarted","Data":"8e76e06207dfded21bbb55575d5f33af500021dcb5440f991b65abbf5cd7d8a5"}
Jul 16 22:21:57.271272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:57.271226    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:57.294392 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:57.294321    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s" podStartSLOduration=1.297632206 podStartE2EDuration="7.294304694s" totalImagesPullingTime="5.996672488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:21:57.293046329 +0000 UTC m=+309.966305341" watchObservedRunningTime="2026-07-16 22:21:57.294304694 +0000 UTC m=+309.967563701"
Jul 16 22:21:57.799142 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:57.799093    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:57.990906 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:57.990870    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:58.258591 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:58.258495    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:58.277433 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:58.277411    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:21:58.377904 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:21:58.377870    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:01.205876 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.205838    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:01.273727 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.273696    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"]
Jul 16 22:22:01.277399 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.277377    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:01.280380 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.280355    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jcc2z\""
Jul 16 22:22:01.281012 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.280988    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 16 22:22:01.299077 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.299050    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"]
Jul 16 22:22:01.417192 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.417164    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a04d53a-c4aa-45e1-a423-df0ba3948bab-cert\") pod \"kserve-controller-manager-d6f4b65b8-8cf7r\" (UID: \"0a04d53a-c4aa-45e1-a423-df0ba3948bab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:01.417362 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.417208    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p84ft\" (UniqueName: \"kubernetes.io/projected/0a04d53a-c4aa-45e1-a423-df0ba3948bab-kube-api-access-p84ft\") pod \"kserve-controller-manager-d6f4b65b8-8cf7r\" (UID: \"0a04d53a-c4aa-45e1-a423-df0ba3948bab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:01.518698 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.518626    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p84ft\" (UniqueName: \"kubernetes.io/projected/0a04d53a-c4aa-45e1-a423-df0ba3948bab-kube-api-access-p84ft\") pod \"kserve-controller-manager-d6f4b65b8-8cf7r\" (UID: \"0a04d53a-c4aa-45e1-a423-df0ba3948bab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:01.518829 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.518720    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a04d53a-c4aa-45e1-a423-df0ba3948bab-cert\") pod \"kserve-controller-manager-d6f4b65b8-8cf7r\" (UID: \"0a04d53a-c4aa-45e1-a423-df0ba3948bab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:01.518829 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:22:01.518814    2556 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 16 22:22:01.518893 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:22:01.518875    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a04d53a-c4aa-45e1-a423-df0ba3948bab-cert podName:0a04d53a-c4aa-45e1-a423-df0ba3948bab nodeName:}" failed. No retries permitted until 2026-07-16 22:22:02.018856927 +0000 UTC m=+314.692115930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0a04d53a-c4aa-45e1-a423-df0ba3948bab-cert") pod "kserve-controller-manager-d6f4b65b8-8cf7r" (UID: "0a04d53a-c4aa-45e1-a423-df0ba3948bab") : secret "kserve-webhook-server-cert" not found
Jul 16 22:22:01.535362 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.535334    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p84ft\" (UniqueName: \"kubernetes.io/projected/0a04d53a-c4aa-45e1-a423-df0ba3948bab-kube-api-access-p84ft\") pod \"kserve-controller-manager-d6f4b65b8-8cf7r\" (UID: \"0a04d53a-c4aa-45e1-a423-df0ba3948bab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:01.536037 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.536007    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:01.639129 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.639088    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:01.736176 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:01.736140    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:02.022957 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:02.022927    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a04d53a-c4aa-45e1-a423-df0ba3948bab-cert\") pod \"kserve-controller-manager-d6f4b65b8-8cf7r\" (UID: \"0a04d53a-c4aa-45e1-a423-df0ba3948bab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:02.025533 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:02.025509    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0a04d53a-c4aa-45e1-a423-df0ba3948bab-cert\") pod \"kserve-controller-manager-d6f4b65b8-8cf7r\" (UID: \"0a04d53a-c4aa-45e1-a423-df0ba3948bab\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:02.189425 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:02.189387    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:02.322795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:02.322773    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"]
Jul 16 22:22:02.324707 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:22:02.324682    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a04d53a_c4aa_45e1_a423_df0ba3948bab.slice/crio-4e7588d1d43e8188e5445c04e5ecb0f3b3e6652e4f0019e07a82d7444b0bc663 WatchSource:0}: Error finding container 4e7588d1d43e8188e5445c04e5ecb0f3b3e6652e4f0019e07a82d7444b0bc663: Status 404 returned error can't find the container with id 4e7588d1d43e8188e5445c04e5ecb0f3b3e6652e4f0019e07a82d7444b0bc663
Jul 16 22:22:02.521574 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:02.521533    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:02.630939 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:02.630850    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:03.162208 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.162174    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"]
Jul 16 22:22:03.162413 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.162378    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s" podUID="48a52781-2011-4433-992e-fb32748f9bfd" containerName="discovery" containerID="cri-o://8e76e06207dfded21bbb55575d5f33af500021dcb5440f991b65abbf5cd7d8a5" gracePeriod=30
Jul 16 22:22:03.300942 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.300899    2556 generic.go:356] "Generic (PLEG): container finished" podID="48a52781-2011-4433-992e-fb32748f9bfd" containerID="8e76e06207dfded21bbb55575d5f33af500021dcb5440f991b65abbf5cd7d8a5" exitCode=0
Jul 16 22:22:03.301085 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.300957    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s" event={"ID":"48a52781-2011-4433-992e-fb32748f9bfd","Type":"ContainerDied","Data":"8e76e06207dfded21bbb55575d5f33af500021dcb5440f991b65abbf5cd7d8a5"}
Jul 16 22:22:03.302593 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.302567    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r" event={"ID":"0a04d53a-c4aa-45e1-a423-df0ba3948bab","Type":"ContainerStarted","Data":"4e7588d1d43e8188e5445c04e5ecb0f3b3e6652e4f0019e07a82d7444b0bc663"}
Jul 16 22:22:03.436253 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.436227    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:22:03.534548 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.534513    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-istio-token\") pod \"48a52781-2011-4433-992e-fb32748f9bfd\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") "
Jul 16 22:22:03.534713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.534605    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a52781-2011-4433-992e-fb32748f9bfd-local-certs\") pod \"48a52781-2011-4433-992e-fb32748f9bfd\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") "
Jul 16 22:22:03.534713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.534648    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-kubeconfig\") pod \"48a52781-2011-4433-992e-fb32748f9bfd\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") "
Jul 16 22:22:03.534713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.534671    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwrdz\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-kube-api-access-gwrdz\") pod \"48a52781-2011-4433-992e-fb32748f9bfd\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") "
Jul 16 22:22:03.534713 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.534704    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-cacerts\") pod \"48a52781-2011-4433-992e-fb32748f9bfd\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") "
Jul 16 22:22:03.534933 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.534732    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-ca-configmap\") pod \"48a52781-2011-4433-992e-fb32748f9bfd\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") "
Jul 16 22:22:03.534933 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.534771    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-dns-cert\") pod \"48a52781-2011-4433-992e-fb32748f9bfd\" (UID: \"48a52781-2011-4433-992e-fb32748f9bfd\") "
Jul 16 22:22:03.535386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.535323    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "48a52781-2011-4433-992e-fb32748f9bfd" (UID: "48a52781-2011-4433-992e-fb32748f9bfd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:22:03.537228 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.537162    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "48a52781-2011-4433-992e-fb32748f9bfd" (UID: "48a52781-2011-4433-992e-fb32748f9bfd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:22:03.537392 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.537369    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "48a52781-2011-4433-992e-fb32748f9bfd" (UID: "48a52781-2011-4433-992e-fb32748f9bfd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:22:03.537518 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.537495    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a52781-2011-4433-992e-fb32748f9bfd-local-certs" (OuterVolumeSpecName: "local-certs") pod "48a52781-2011-4433-992e-fb32748f9bfd" (UID: "48a52781-2011-4433-992e-fb32748f9bfd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:22:03.537518 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.537505    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-istio-token" (OuterVolumeSpecName: "istio-token") pod "48a52781-2011-4433-992e-fb32748f9bfd" (UID: "48a52781-2011-4433-992e-fb32748f9bfd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:22:03.537652 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.537637    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-kube-api-access-gwrdz" (OuterVolumeSpecName: "kube-api-access-gwrdz") pod "48a52781-2011-4433-992e-fb32748f9bfd" (UID: "48a52781-2011-4433-992e-fb32748f9bfd"). InnerVolumeSpecName "kube-api-access-gwrdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:22:03.537744 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.537723    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-cacerts" (OuterVolumeSpecName: "cacerts") pod "48a52781-2011-4433-992e-fb32748f9bfd" (UID: "48a52781-2011-4433-992e-fb32748f9bfd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:22:03.636317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.636288    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-istio-token\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:03.636317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.636314    2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/48a52781-2011-4433-992e-fb32748f9bfd-local-certs\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:03.636317 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.636322    2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-kubeconfig\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:03.636539 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.636331    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwrdz\" (UniqueName: \"kubernetes.io/projected/48a52781-2011-4433-992e-fb32748f9bfd-kube-api-access-gwrdz\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:03.636539 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.636340    2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-cacerts\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:03.636539 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.636349    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-ca-configmap\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:03.636539 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:03.636358    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/48a52781-2011-4433-992e-fb32748f9bfd-istio-csr-dns-cert\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:04.309125 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:04.309076    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"
Jul 16 22:22:04.309293 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:04.309086    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s" event={"ID":"48a52781-2011-4433-992e-fb32748f9bfd","Type":"ContainerDied","Data":"d092473676e5150005e589a88759c6823e1e35b63ba90bf97fe11a1de407edf6"}
Jul 16 22:22:04.309293 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:04.309241    2556 scope.go:117] "RemoveContainer" containerID="8e76e06207dfded21bbb55575d5f33af500021dcb5440f991b65abbf5cd7d8a5"
Jul 16 22:22:04.327761 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:04.327734    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"]
Jul 16 22:22:04.331684 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:04.331656    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lwf4s"]
Jul 16 22:22:05.320241 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:05.320192    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r" event={"ID":"0a04d53a-c4aa-45e1-a423-df0ba3948bab","Type":"ContainerStarted","Data":"f66de5121ae02fc75b844b9fb60483f556fda9400ae99bf54b982cc1b6bc2902"}
Jul 16 22:22:05.321313 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:05.321101    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:05.352579 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:05.352523    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r" podStartSLOduration=1.8867685220000001 podStartE2EDuration="4.352508101s" totalImagesPullingTime="2.465739579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:05.351586975 +0000 UTC m=+318.024845982" watchObservedRunningTime="2026-07-16 22:22:05.352508101 +0000 UTC m=+318.025767086"
Jul 16 22:22:05.810948 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:05.810912    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a52781-2011-4433-992e-fb32748f9bfd" path="/var/lib/kubelet/pods/48a52781-2011-4433-992e-fb32748f9bfd/volumes"
Jul 16 22:22:06.267927 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:06.267846    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ng8d4"
Jul 16 22:22:06.269452 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:06.269431    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nlf8k"
Jul 16 22:22:19.510727 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:19.510677    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:19.568546 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:19.568512    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:37.338249 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:37.338211    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8cf7r"
Jul 16 22:22:46.128932 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:46.128846    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:49.981253 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:49.981208    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:50.069910 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.069869    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:50.106432 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.106400    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwxn8"]
Jul 16 22:22:50.107564 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.107515    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a52781-2011-4433-992e-fb32748f9bfd" containerName="discovery"
Jul 16 22:22:50.107714 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.107568    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a52781-2011-4433-992e-fb32748f9bfd" containerName="discovery"
Jul 16 22:22:50.107714 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.107678    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a52781-2011-4433-992e-fb32748f9bfd" containerName="discovery"
Jul 16 22:22:50.110614 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.110588    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8"
Jul 16 22:22:50.113582 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.113533    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-dspk5\""
Jul 16 22:22:50.114090 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.114067    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 22:22:50.114221 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.114202    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwxn8"]
Jul 16 22:22:50.114324 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.114300    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 22:22:50.169246 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.169207    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:50.239397 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.239311    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss8xw\" (UniqueName: \"kubernetes.io/projected/b1741903-2ef8-43b0-bebf-a034256b3dd4-kube-api-access-ss8xw\") pod \"kuadrant-operator-catalog-dwxn8\" (UID: \"b1741903-2ef8-43b0-bebf-a034256b3dd4\") " pod="kuadrant-system/kuadrant-operator-catalog-dwxn8"
Jul 16 22:22:50.340255 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.340220    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss8xw\" (UniqueName: \"kubernetes.io/projected/b1741903-2ef8-43b0-bebf-a034256b3dd4-kube-api-access-ss8xw\") pod \"kuadrant-operator-catalog-dwxn8\" (UID: \"b1741903-2ef8-43b0-bebf-a034256b3dd4\") " pod="kuadrant-system/kuadrant-operator-catalog-dwxn8"
Jul 16 22:22:50.350551 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.350523    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss8xw\" (UniqueName: \"kubernetes.io/projected/b1741903-2ef8-43b0-bebf-a034256b3dd4-kube-api-access-ss8xw\") pod \"kuadrant-operator-catalog-dwxn8\" (UID: \"b1741903-2ef8-43b0-bebf-a034256b3dd4\") " pod="kuadrant-system/kuadrant-operator-catalog-dwxn8"
Jul 16 22:22:50.422520 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.422484    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8"
Jul 16 22:22:50.462449 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.462421    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwxn8"]
Jul 16 22:22:50.587287 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:22:50.587240    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1741903_2ef8_43b0_bebf_a034256b3dd4.slice/crio-c1c92542f32346a4f89bc40ebb5b5bea0d75445b4ac2271cdf16ec4048d4608d WatchSource:0}: Error finding container c1c92542f32346a4f89bc40ebb5b5bea0d75445b4ac2271cdf16ec4048d4608d: Status 404 returned error can't find the container with id c1c92542f32346a4f89bc40ebb5b5bea0d75445b4ac2271cdf16ec4048d4608d
Jul 16 22:22:50.588044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.588017    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwxn8"]
Jul 16 22:22:50.668135 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.668072    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jt5wq"]
Jul 16 22:22:50.671205 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.671184    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:22:50.682693 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.682672    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jt5wq"]
Jul 16 22:22:50.743853 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.743822    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqmwk\" (UniqueName: \"kubernetes.io/projected/644e0ca1-409b-4252-bcee-f13517f84d13-kube-api-access-pqmwk\") pod \"kuadrant-operator-catalog-jt5wq\" (UID: \"644e0ca1-409b-4252-bcee-f13517f84d13\") " pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:22:50.844662 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.844632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqmwk\" (UniqueName: \"kubernetes.io/projected/644e0ca1-409b-4252-bcee-f13517f84d13-kube-api-access-pqmwk\") pod \"kuadrant-operator-catalog-jt5wq\" (UID: \"644e0ca1-409b-4252-bcee-f13517f84d13\") " pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:22:50.855873 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.855845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqmwk\" (UniqueName: \"kubernetes.io/projected/644e0ca1-409b-4252-bcee-f13517f84d13-kube-api-access-pqmwk\") pod \"kuadrant-operator-catalog-jt5wq\" (UID: \"644e0ca1-409b-4252-bcee-f13517f84d13\") " pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:22:50.980385 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:50.980352    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:22:51.129294 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:51.129220    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jt5wq"]
Jul 16 22:22:51.132144 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:22:51.132096    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644e0ca1_409b_4252_bcee_f13517f84d13.slice/crio-e91d0219f4f041411e8884ea08602bf004ed2a9c8c17eefba04b25273b042870 WatchSource:0}: Error finding container e91d0219f4f041411e8884ea08602bf004ed2a9c8c17eefba04b25273b042870: Status 404 returned error can't find the container with id e91d0219f4f041411e8884ea08602bf004ed2a9c8c17eefba04b25273b042870
Jul 16 22:22:51.256720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:51.256684    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:51.359940 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:51.359880    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:51.535606 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:51.535568    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq" event={"ID":"644e0ca1-409b-4252-bcee-f13517f84d13","Type":"ContainerStarted","Data":"e91d0219f4f041411e8884ea08602bf004ed2a9c8c17eefba04b25273b042870"}
Jul 16 22:22:51.536642 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:51.536619    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8" event={"ID":"b1741903-2ef8-43b0-bebf-a034256b3dd4","Type":"ContainerStarted","Data":"c1c92542f32346a4f89bc40ebb5b5bea0d75445b4ac2271cdf16ec4048d4608d"}
Jul 16 22:22:53.547276 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.547244    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq" event={"ID":"644e0ca1-409b-4252-bcee-f13517f84d13","Type":"ContainerStarted","Data":"b3de867bd7d6b77f5e9f5e648f7c2014284b5e0c311b910a7c0458923f55f847"}
Jul 16 22:22:53.548685 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.548662    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8" event={"ID":"b1741903-2ef8-43b0-bebf-a034256b3dd4","Type":"ContainerStarted","Data":"bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741"}
Jul 16 22:22:53.548784 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.548749    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8" podUID="b1741903-2ef8-43b0-bebf-a034256b3dd4" containerName="registry-server" containerID="cri-o://bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741" gracePeriod=2
Jul 16 22:22:53.565402 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.565364    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq" podStartSLOduration=1.959676167 podStartE2EDuration="3.565350179s" totalImagesPullingTime="1.605674012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:53.564713385 +0000 UTC m=+366.237972392" watchObservedRunningTime="2026-07-16 22:22:53.565350179 +0000 UTC m=+366.238609180"
Jul 16 22:22:53.581718 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.581678    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8" podStartSLOduration=1.434628456 podStartE2EDuration="3.58166434s" totalImagesPullingTime="2.147035884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:53.580426084 +0000 UTC m=+366.253685092" watchObservedRunningTime="2026-07-16 22:22:53.58166434 +0000 UTC m=+366.254923348"
Jul 16 22:22:53.792590 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.792565    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8"
Jul 16 22:22:53.871319 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.871248    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss8xw\" (UniqueName: \"kubernetes.io/projected/b1741903-2ef8-43b0-bebf-a034256b3dd4-kube-api-access-ss8xw\") pod \"b1741903-2ef8-43b0-bebf-a034256b3dd4\" (UID: \"b1741903-2ef8-43b0-bebf-a034256b3dd4\") "
Jul 16 22:22:53.873462 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.873437    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1741903-2ef8-43b0-bebf-a034256b3dd4-kube-api-access-ss8xw" (OuterVolumeSpecName: "kube-api-access-ss8xw") pod "b1741903-2ef8-43b0-bebf-a034256b3dd4" (UID: "b1741903-2ef8-43b0-bebf-a034256b3dd4"). InnerVolumeSpecName "kube-api-access-ss8xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:22:53.971986 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:53.971946    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss8xw\" (UniqueName: \"kubernetes.io/projected/b1741903-2ef8-43b0-bebf-a034256b3dd4-kube-api-access-ss8xw\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:22:54.554542 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.554508    2556 generic.go:356] "Generic (PLEG): container finished" podID="b1741903-2ef8-43b0-bebf-a034256b3dd4" containerID="bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741" exitCode=0
Jul 16 22:22:54.554976 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.554550    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8"
Jul 16 22:22:54.554976 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.554585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8" event={"ID":"b1741903-2ef8-43b0-bebf-a034256b3dd4","Type":"ContainerDied","Data":"bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741"}
Jul 16 22:22:54.554976 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.554623    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dwxn8" event={"ID":"b1741903-2ef8-43b0-bebf-a034256b3dd4","Type":"ContainerDied","Data":"c1c92542f32346a4f89bc40ebb5b5bea0d75445b4ac2271cdf16ec4048d4608d"}
Jul 16 22:22:54.554976 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.554642    2556 scope.go:117] "RemoveContainer" containerID="bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741"
Jul 16 22:22:54.563386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.563370    2556 scope.go:117] "RemoveContainer" containerID="bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741"
Jul 16 22:22:54.563654 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:22:54.563636    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741\": container with ID starting with bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741 not found: ID does not exist" containerID="bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741"
Jul 16 22:22:54.563705 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.563663    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741"} err="failed to get container status \"bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741\": rpc error: code = NotFound desc = could not find container \"bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741\": container with ID starting with bf13f5b9d06f8408eb956fb9a5a6a00dc731cc3f6914d92005dc34f620b09741 not found: ID does not exist"
Jul 16 22:22:54.577524 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.577503    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwxn8"]
Jul 16 22:22:54.581046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:54.581025    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dwxn8"]
Jul 16 22:22:55.811025 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:22:55.810988    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1741903-2ef8-43b0-bebf-a034256b3dd4" path="/var/lib/kubelet/pods/b1741903-2ef8-43b0-bebf-a034256b3dd4/volumes"
Jul 16 22:23:00.980897 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:00.980839    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:23:00.981402 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:00.980919    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:23:01.003546 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:01.003519    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:23:01.608969 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:01.608941    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jt5wq"
Jul 16 22:23:18.866948 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:18.866909    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:21.505325 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.505290    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"]
Jul 16 22:23:21.505709 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.505610    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1741903-2ef8-43b0-bebf-a034256b3dd4" containerName="registry-server"
Jul 16 22:23:21.505709 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.505621    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1741903-2ef8-43b0-bebf-a034256b3dd4" containerName="registry-server"
Jul 16 22:23:21.505709 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.505688    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1741903-2ef8-43b0-bebf-a034256b3dd4" containerName="registry-server"
Jul 16 22:23:21.507729 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.507711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"
Jul 16 22:23:21.510068 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.510045    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 16 22:23:21.510202 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.510140    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lz4vc\""
Jul 16 22:23:21.522750 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.522730    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"]
Jul 16 22:23:21.699091 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.699052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5brcz\" (UniqueName: \"kubernetes.io/projected/54dd2528-227f-42ed-86a4-a8555cdeec0e-kube-api-access-5brcz\") pod \"dns-operator-controller-manager-864dd78c4c-mz287\" (UID: \"54dd2528-227f-42ed-86a4-a8555cdeec0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"
Jul 16 22:23:21.799870 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.799833    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5brcz\" (UniqueName: \"kubernetes.io/projected/54dd2528-227f-42ed-86a4-a8555cdeec0e-kube-api-access-5brcz\") pod \"dns-operator-controller-manager-864dd78c4c-mz287\" (UID: \"54dd2528-227f-42ed-86a4-a8555cdeec0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"
Jul 16 22:23:21.812496 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.812470    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5brcz\" (UniqueName: \"kubernetes.io/projected/54dd2528-227f-42ed-86a4-a8555cdeec0e-kube-api-access-5brcz\") pod \"dns-operator-controller-manager-864dd78c4c-mz287\" (UID: \"54dd2528-227f-42ed-86a4-a8555cdeec0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"
Jul 16 22:23:21.818234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.818216    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"
Jul 16 22:23:21.951323 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:21.951213    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"]
Jul 16 22:23:21.954197 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:23:21.954169    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54dd2528_227f_42ed_86a4_a8555cdeec0e.slice/crio-7bf1a7f203ca35d8740697403028457184c62c0721940ed08f8d15b854364e42 WatchSource:0}: Error finding container 7bf1a7f203ca35d8740697403028457184c62c0721940ed08f8d15b854364e42: Status 404 returned error can't find the container with id 7bf1a7f203ca35d8740697403028457184c62c0721940ed08f8d15b854364e42
Jul 16 22:23:22.677273 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:22.677236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287" event={"ID":"54dd2528-227f-42ed-86a4-a8555cdeec0e","Type":"ContainerStarted","Data":"7bf1a7f203ca35d8740697403028457184c62c0721940ed08f8d15b854364e42"}
Jul 16 22:23:23.068433 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:23.068365    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:24.688048 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:24.688012    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287" event={"ID":"54dd2528-227f-42ed-86a4-a8555cdeec0e","Type":"ContainerStarted","Data":"5f1dddd144c2c56e6afecb6102c735e1c723c0f6594ae51b0a6ee082085e8fca"}
Jul 16 22:23:24.688691 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:24.688165    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"
Jul 16 22:23:24.705031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:24.704983    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287" podStartSLOduration=1.5079372960000001 podStartE2EDuration="3.704972238s" totalImagesPullingTime="2.197034942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:23:24.704739216 +0000 UTC m=+397.377998226" watchObservedRunningTime="2026-07-16 22:23:24.704972238 +0000 UTC m=+397.378231245"
Jul 16 22:23:24.928136 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:24.928094    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"]
Jul 16 22:23:24.931640 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:24.931617    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:24.933878 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:24.933861    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9dkgl\""
Jul 16 22:23:24.942903 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:24.942848    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"]
Jul 16 22:23:25.022782 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.022753    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9lf8\" (UniqueName: \"kubernetes.io/projected/deb88ce1-72fb-4c04-ad69-6ff563a2430c-kube-api-access-r9lf8\") pod \"limitador-operator-controller-manager-859d7b78bd-blp4p\" (UID: \"deb88ce1-72fb-4c04-ad69-6ff563a2430c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:25.123968 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.123936    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9lf8\" (UniqueName: \"kubernetes.io/projected/deb88ce1-72fb-4c04-ad69-6ff563a2430c-kube-api-access-r9lf8\") pod \"limitador-operator-controller-manager-859d7b78bd-blp4p\" (UID: \"deb88ce1-72fb-4c04-ad69-6ff563a2430c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:25.141384 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.141356    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9lf8\" (UniqueName: \"kubernetes.io/projected/deb88ce1-72fb-4c04-ad69-6ff563a2430c-kube-api-access-r9lf8\") pod \"limitador-operator-controller-manager-859d7b78bd-blp4p\" (UID: \"deb88ce1-72fb-4c04-ad69-6ff563a2430c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:25.241265 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.241178    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:25.372870 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.372844    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"]
Jul 16 22:23:25.374751 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:23:25.374718    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb88ce1_72fb_4c04_ad69_6ff563a2430c.slice/crio-6a386d360f370342ed596ebcb0cf99c0eed44ad6b49a7bfba8a3ac4e1ec56ddf WatchSource:0}: Error finding container 6a386d360f370342ed596ebcb0cf99c0eed44ad6b49a7bfba8a3ac4e1ec56ddf: Status 404 returned error can't find the container with id 6a386d360f370342ed596ebcb0cf99c0eed44ad6b49a7bfba8a3ac4e1ec56ddf
Jul 16 22:23:25.558494 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.558461    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:25.673663 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.673628    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:25.693044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:25.693013    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p" event={"ID":"deb88ce1-72fb-4c04-ad69-6ff563a2430c","Type":"ContainerStarted","Data":"6a386d360f370342ed596ebcb0cf99c0eed44ad6b49a7bfba8a3ac4e1ec56ddf"}
Jul 16 22:23:26.916012 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:26.915973    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:27.705099 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:27.705063    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p" event={"ID":"deb88ce1-72fb-4c04-ad69-6ff563a2430c","Type":"ContainerStarted","Data":"a91fbece4c230e54cff83e07d519e65aa9da53dc212292d0d4c8a79a07201851"}
Jul 16 22:23:27.705297 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:27.705169    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:27.730970 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:27.730923    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p" podStartSLOduration=2.169525913 podStartE2EDuration="3.730910251s" totalImagesPullingTime="1.561384338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:23:27.729026816 +0000 UTC m=+400.402285823" watchObservedRunningTime="2026-07-16 22:23:27.730910251 +0000 UTC m=+400.404169259"
Jul 16 22:23:29.936607 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:29.936574    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"]
Jul 16 22:23:29.940024 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:29.940007    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:29.943298 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:29.943277    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-shwv8\""
Jul 16 22:23:29.952993 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:29.952969    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"]
Jul 16 22:23:29.962943 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:29.962924    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a2f8803-edab-4237-9b9d-2c2ade6b179d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:29.963194 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:29.962971    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9wrj\" (UniqueName: \"kubernetes.io/projected/8a2f8803-edab-4237-9b9d-2c2ade6b179d-kube-api-access-n9wrj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:30.064368 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.064338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a2f8803-edab-4237-9b9d-2c2ade6b179d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:30.064505 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.064402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9wrj\" (UniqueName: \"kubernetes.io/projected/8a2f8803-edab-4237-9b9d-2c2ade6b179d-kube-api-access-n9wrj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:30.064771 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.064749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a2f8803-edab-4237-9b9d-2c2ade6b179d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:30.065965 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.065943    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:30.073886 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.073867    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9wrj\" (UniqueName: \"kubernetes.io/projected/8a2f8803-edab-4237-9b9d-2c2ade6b179d-kube-api-access-n9wrj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:30.250006 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.249923    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:30.409638 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.409597    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"]
Jul 16 22:23:30.411502 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:23:30.411475    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2f8803_edab_4237_9b9d_2c2ade6b179d.slice/crio-8d5643809636028a79ef24d9bf36136858b2ed850c8c32f2fe72c3734c89da39 WatchSource:0}: Error finding container 8d5643809636028a79ef24d9bf36136858b2ed850c8c32f2fe72c3734c89da39: Status 404 returned error can't find the container with id 8d5643809636028a79ef24d9bf36136858b2ed850c8c32f2fe72c3734c89da39
Jul 16 22:23:30.485002 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.484973    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:30.725361 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.725326    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj" event={"ID":"8a2f8803-edab-4237-9b9d-2c2ade6b179d","Type":"ContainerStarted","Data":"8d5643809636028a79ef24d9bf36136858b2ed850c8c32f2fe72c3734c89da39"}
Jul 16 22:23:30.803975 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.803941    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:30.954275 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:30.954243    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:33.146953 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.146914    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"]
Jul 16 22:23:33.150533 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.150510    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"
Jul 16 22:23:33.155327 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.155305    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-2lnfk\""
Jul 16 22:23:33.165528 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.165503    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"]
Jul 16 22:23:33.189258 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.189223    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9mfw\" (UniqueName: \"kubernetes.io/projected/aca7c23d-c1f0-40e5-8048-b3d8b62b9a02-kube-api-access-v9mfw\") pod \"authorino-operator-75cc8d5cb7-sg4d9\" (UID: \"aca7c23d-c1f0-40e5-8048-b3d8b62b9a02\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"
Jul 16 22:23:33.290056 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.290023    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9mfw\" (UniqueName: \"kubernetes.io/projected/aca7c23d-c1f0-40e5-8048-b3d8b62b9a02-kube-api-access-v9mfw\") pod \"authorino-operator-75cc8d5cb7-sg4d9\" (UID: \"aca7c23d-c1f0-40e5-8048-b3d8b62b9a02\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"
Jul 16 22:23:33.298911 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.298880    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9mfw\" (UniqueName: \"kubernetes.io/projected/aca7c23d-c1f0-40e5-8048-b3d8b62b9a02-kube-api-access-v9mfw\") pod \"authorino-operator-75cc8d5cb7-sg4d9\" (UID: \"aca7c23d-c1f0-40e5-8048-b3d8b62b9a02\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"
Jul 16 22:23:33.460819 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.460741    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"
Jul 16 22:23:33.545346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.545307    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:33.596163 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.596132    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"]
Jul 16 22:23:33.599707 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:23:33.599677    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca7c23d_c1f0_40e5_8048_b3d8b62b9a02.slice/crio-5150e4503a7c9659eb4136cc0ab92a9ce6ac4cbdc9a87c69e4b5a0808a301587 WatchSource:0}: Error finding container 5150e4503a7c9659eb4136cc0ab92a9ce6ac4cbdc9a87c69e4b5a0808a301587: Status 404 returned error can't find the container with id 5150e4503a7c9659eb4136cc0ab92a9ce6ac4cbdc9a87c69e4b5a0808a301587
Jul 16 22:23:33.740337 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:33.740244    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9" event={"ID":"aca7c23d-c1f0-40e5-8048-b3d8b62b9a02","Type":"ContainerStarted","Data":"5150e4503a7c9659eb4136cc0ab92a9ce6ac4cbdc9a87c69e4b5a0808a301587"}
Jul 16 22:23:34.892367 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:34.892330    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:34.995266 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:34.995229    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:35.695758 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:35.695724    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mz287"
Jul 16 22:23:37.767361 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:37.767303    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj" event={"ID":"8a2f8803-edab-4237-9b9d-2c2ade6b179d","Type":"ContainerStarted","Data":"7d70cd5640bfe986862b6cf7e0fead25c065897574c4f63f7395076865357192"}
Jul 16 22:23:37.767843 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:37.767403    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:37.768830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:37.768806    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9" event={"ID":"aca7c23d-c1f0-40e5-8048-b3d8b62b9a02","Type":"ContainerStarted","Data":"4aa1fbfa639d9c7f62334fc8c32f7ff295eaee789f11ea965619c7279f32e995"}
Jul 16 22:23:37.768943 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:37.768928    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"
Jul 16 22:23:37.796335 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:37.796288    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj" podStartSLOduration=2.072425212 podStartE2EDuration="8.796272525s" totalImagesPullingTime="6.723847313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:23:37.792674149 +0000 UTC m=+410.465933156" watchObservedRunningTime="2026-07-16 22:23:37.796272525 +0000 UTC m=+410.469531532"
Jul 16 22:23:37.813695 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:37.813641    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9" podStartSLOduration=1.288345249 podStartE2EDuration="4.813617488s" totalImagesPullingTime="3.525272239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:23:37.81289522 +0000 UTC m=+410.486154241" watchObservedRunningTime="2026-07-16 22:23:37.813617488 +0000 UTC m=+410.486876496"
Jul 16 22:23:38.681217 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:38.681179    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:38.717722 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:38.717696    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:38.785276 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:38.785242    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:40.050353 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.050320    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"]
Jul 16 22:23:40.053747 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.053728    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.056279 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.056261    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 16 22:23:40.056786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.056769    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 16 22:23:40.056889 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.056774    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r9mf7\""
Jul 16 22:23:40.067589 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.067569    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"]
Jul 16 22:23:40.073219 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.073195    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:40.151754 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.151729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9721a53-7880-477d-bf57-aeef00b239d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.151897 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.151767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a9721a53-7880-477d-bf57-aeef00b239d0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.151897 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.151796    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk224\" (UniqueName: \"kubernetes.io/projected/a9721a53-7880-477d-bf57-aeef00b239d0-kube-api-access-pk224\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.253205 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.253173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk224\" (UniqueName: \"kubernetes.io/projected/a9721a53-7880-477d-bf57-aeef00b239d0-kube-api-access-pk224\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.253409 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.253276    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9721a53-7880-477d-bf57-aeef00b239d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.253409 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.253311    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a9721a53-7880-477d-bf57-aeef00b239d0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.253539 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:40.253411    2556 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 16 22:23:40.253539 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:40.253490    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9721a53-7880-477d-bf57-aeef00b239d0-plugin-serving-cert podName:a9721a53-7880-477d-bf57-aeef00b239d0 nodeName:}" failed. No retries permitted until 2026-07-16 22:23:40.753468945 +0000 UTC m=+413.426727942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a9721a53-7880-477d-bf57-aeef00b239d0-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-zprlg" (UID: "a9721a53-7880-477d-bf57-aeef00b239d0") : secret "plugin-serving-cert" not found
Jul 16 22:23:40.253992 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.253969    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a9721a53-7880-477d-bf57-aeef00b239d0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.263163 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.263139    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk224\" (UniqueName: \"kubernetes.io/projected/a9721a53-7880-477d-bf57-aeef00b239d0-kube-api-access-pk224\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.401264 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.401180    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:40.756796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.756719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9721a53-7880-477d-bf57-aeef00b239d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.759292 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.759262    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9721a53-7880-477d-bf57-aeef00b239d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zprlg\" (UID: \"a9721a53-7880-477d-bf57-aeef00b239d0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:40.858024 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.857990    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:40.928300 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.928268    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:40.963433 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:40.963406    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"
Jul 16 22:23:41.101496 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:41.101474    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg"]
Jul 16 22:23:41.104178 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:23:41.104146    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9721a53_7880_477d_bf57_aeef00b239d0.slice/crio-7cdc6ee22a30cf325d1a5d2a64e951072e28ad0842cd9f758fb6ea59b9dc87a0 WatchSource:0}: Error finding container 7cdc6ee22a30cf325d1a5d2a64e951072e28ad0842cd9f758fb6ea59b9dc87a0: Status 404 returned error can't find the container with id 7cdc6ee22a30cf325d1a5d2a64e951072e28ad0842cd9f758fb6ea59b9dc87a0
Jul 16 22:23:41.797074 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:41.797038    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg" event={"ID":"a9721a53-7880-477d-bf57-aeef00b239d0","Type":"ContainerStarted","Data":"7cdc6ee22a30cf325d1a5d2a64e951072e28ad0842cd9f758fb6ea59b9dc87a0"}
Jul 16 22:23:42.068329 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:42.068243    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:42.138565 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:42.138531    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:48.781388 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:48.781352    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sg4d9"
Jul 16 22:23:48.781851 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:48.781535    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:50.513797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.513763    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:50.541647 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.541585    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"]
Jul 16 22:23:50.541857 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.541812    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" containerName="manager" containerID="cri-o://7d70cd5640bfe986862b6cf7e0fead25c065897574c4f63f7395076865357192" gracePeriod=2
Jul 16 22:23:50.575316 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.575284    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"]
Jul 16 22:23:50.596373 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:50.596334    2556 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8a2f8803-edab-4237-9b9d-2c2ade6b179d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-89mwj\": pods \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:50.602439 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.601682    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"]
Jul 16 22:23:50.602439 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.601919    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" containerName="manager" containerID="cri-o://a91fbece4c230e54cff83e07d519e65aa9da53dc212292d0d4c8a79a07201851" gracePeriod=2
Jul 16 22:23:50.604488 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.604446    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"]
Jul 16 22:23:50.604604 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:50.604523    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:50.605067 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.605037    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" containerName="manager"
Jul 16 22:23:50.605067 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.605059    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" containerName="manager"
Jul 16 22:23:50.605460 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.605200    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" containerName="manager"
Jul 16 22:23:50.613647 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.613626    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"]
Jul 16 22:23:50.613793 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.613769    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.630057 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.630031    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"]
Jul 16 22:23:50.637864 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.637834    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"]
Jul 16 22:23:50.638429 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.638403    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" containerName="manager"
Jul 16 22:23:50.638429 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.638429    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" containerName="manager"
Jul 16 22:23:50.638591 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.638541    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" containerName="manager"
Jul 16 22:23:50.642918 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.642899    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"
Jul 16 22:23:50.651324 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:50.651187    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blp4p\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:50.655133 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:50.655078    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:50.656975 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.656940    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"]
Jul 16 22:23:50.657092 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:50.657068    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blp4p\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:23:50.659400 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:23:50.659238    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:23:50.660086 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.660058    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:50.749708 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.749666    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9131e364-10d1-4712-8a8b-2354cb410a44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hlb2l\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.749948 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.749749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbsd9\" (UniqueName: \"kubernetes.io/projected/2402b484-8810-402d-ab88-5276cc81922a-kube-api-access-bbsd9\") pod \"limitador-operator-controller-manager-859d7b78bd-9488z\" (UID: \"2402b484-8810-402d-ab88-5276cc81922a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"
Jul 16 22:23:50.749948 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.749805    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bk5k\" (UniqueName: \"kubernetes.io/projected/9131e364-10d1-4712-8a8b-2354cb410a44-kube-api-access-5bk5k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hlb2l\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.851053 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.851022    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9131e364-10d1-4712-8a8b-2354cb410a44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hlb2l\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.851281 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.851092    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbsd9\" (UniqueName: \"kubernetes.io/projected/2402b484-8810-402d-ab88-5276cc81922a-kube-api-access-bbsd9\") pod \"limitador-operator-controller-manager-859d7b78bd-9488z\" (UID: \"2402b484-8810-402d-ab88-5276cc81922a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"
Jul 16 22:23:50.851281 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.851202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bk5k\" (UniqueName: \"kubernetes.io/projected/9131e364-10d1-4712-8a8b-2354cb410a44-kube-api-access-5bk5k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hlb2l\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.851484 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.851460    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9131e364-10d1-4712-8a8b-2354cb410a44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hlb2l\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.863685 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.863632    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbsd9\" (UniqueName: \"kubernetes.io/projected/2402b484-8810-402d-ab88-5276cc81922a-kube-api-access-bbsd9\") pod \"limitador-operator-controller-manager-859d7b78bd-9488z\" (UID: \"2402b484-8810-402d-ab88-5276cc81922a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"
Jul 16 22:23:50.865793 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.865769    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bk5k\" (UniqueName: \"kubernetes.io/projected/9131e364-10d1-4712-8a8b-2354cb410a44-kube-api-access-5bk5k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hlb2l\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.969040 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.968991    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:23:50.977626 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:50.977599    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"
Jul 16 22:23:51.844643 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:51.844599    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:51.942221 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:51.942169    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:52.043552 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:52.043501    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:53.165277 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:53.165217    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:53.351087 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:53.351049    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:53.813617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:53.813581    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:53.909481 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:53.909432    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:55.949832 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:55.949790    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:56.051668 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:56.051630    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:59.512647 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:59.512610    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:59.614584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:23:59.614541    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:03.928970 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:03.928912    2556 generic.go:356] "Generic (PLEG): container finished" podID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" containerID="a91fbece4c230e54cff83e07d519e65aa9da53dc212292d0d4c8a79a07201851" exitCode=0
Jul 16 22:24:03.931776 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:03.931413    2556 generic.go:356] "Generic (PLEG): container finished" podID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" containerID="7d70cd5640bfe986862b6cf7e0fead25c065897574c4f63f7395076865357192" exitCode=0
Jul 16 22:24:03.934910 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:03.934888    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:24:03.937170 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:03.937125    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blp4p\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:24:03.945422 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:03.945400    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:24:03.947703 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:03.947655    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:24:03.950160 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:03.950029    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blp4p\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:24:03.953379 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:03.953358    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"]
Jul 16 22:24:03.955707 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:24:03.955680    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2402b484_8810_402d_ab88_5276cc81922a.slice/crio-6a60b9c9ac9837eb3664404556c4f435b7959f0547366b832c4d566f51b10871 WatchSource:0}: Error finding container 6a60b9c9ac9837eb3664404556c4f435b7959f0547366b832c4d566f51b10871: Status 404 returned error can't find the container with id 6a60b9c9ac9837eb3664404556c4f435b7959f0547366b832c4d566f51b10871
Jul 16 22:24:03.987041 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:03.987020    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"]
Jul 16 22:24:03.989093 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:24:03.989062    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9131e364_10d1_4712_8a8b_2354cb410a44.slice/crio-4d7a57d7f13209dcbaeb8edfb5f0ec371afe988827b1784ff6af0dadea4b345a WatchSource:0}: Error finding container 4d7a57d7f13209dcbaeb8edfb5f0ec371afe988827b1784ff6af0dadea4b345a: Status 404 returned error can't find the container with id 4d7a57d7f13209dcbaeb8edfb5f0ec371afe988827b1784ff6af0dadea4b345a
Jul 16 22:24:04.081957 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.081920    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9wrj\" (UniqueName: \"kubernetes.io/projected/8a2f8803-edab-4237-9b9d-2c2ade6b179d-kube-api-access-n9wrj\") pod \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") "
Jul 16 22:24:04.081957 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.081962    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9lf8\" (UniqueName: \"kubernetes.io/projected/deb88ce1-72fb-4c04-ad69-6ff563a2430c-kube-api-access-r9lf8\") pod \"deb88ce1-72fb-4c04-ad69-6ff563a2430c\" (UID: \"deb88ce1-72fb-4c04-ad69-6ff563a2430c\") "
Jul 16 22:24:04.082229 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.081991    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a2f8803-edab-4237-9b9d-2c2ade6b179d-extensions-socket-volume\") pod \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\" (UID: \"8a2f8803-edab-4237-9b9d-2c2ade6b179d\") "
Jul 16 22:24:04.082447 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.082416    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2f8803-edab-4237-9b9d-2c2ade6b179d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8a2f8803-edab-4237-9b9d-2c2ade6b179d" (UID: "8a2f8803-edab-4237-9b9d-2c2ade6b179d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:24:04.084268 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.084244    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2f8803-edab-4237-9b9d-2c2ade6b179d-kube-api-access-n9wrj" (OuterVolumeSpecName: "kube-api-access-n9wrj") pod "8a2f8803-edab-4237-9b9d-2c2ade6b179d" (UID: "8a2f8803-edab-4237-9b9d-2c2ade6b179d"). InnerVolumeSpecName "kube-api-access-n9wrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:24:04.084329 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.084280    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deb88ce1-72fb-4c04-ad69-6ff563a2430c-kube-api-access-r9lf8" (OuterVolumeSpecName: "kube-api-access-r9lf8") pod "deb88ce1-72fb-4c04-ad69-6ff563a2430c" (UID: "deb88ce1-72fb-4c04-ad69-6ff563a2430c"). InnerVolumeSpecName "kube-api-access-r9lf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:24:04.183250 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.183214    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9wrj\" (UniqueName: \"kubernetes.io/projected/8a2f8803-edab-4237-9b9d-2c2ade6b179d-kube-api-access-n9wrj\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:24:04.183250 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.183242    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9lf8\" (UniqueName: \"kubernetes.io/projected/deb88ce1-72fb-4c04-ad69-6ff563a2430c-kube-api-access-r9lf8\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:24:04.183250 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.183250    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a2f8803-edab-4237-9b9d-2c2ade6b179d-extensions-socket-volume\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:24:04.938691 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.938650    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l" event={"ID":"9131e364-10d1-4712-8a8b-2354cb410a44","Type":"ContainerStarted","Data":"0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951"}
Jul 16 22:24:04.939103 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.938712    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l" event={"ID":"9131e364-10d1-4712-8a8b-2354cb410a44","Type":"ContainerStarted","Data":"4d7a57d7f13209dcbaeb8edfb5f0ec371afe988827b1784ff6af0dadea4b345a"}
Jul 16 22:24:04.939103 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.938735    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:24:04.940291 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.940259    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z" event={"ID":"2402b484-8810-402d-ab88-5276cc81922a","Type":"ContainerStarted","Data":"6296b52f75cebdc5b6cb57142dac0741e2e628730ba969b6aadcab7693487431"}
Jul 16 22:24:04.940445 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.940312    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z" event={"ID":"2402b484-8810-402d-ab88-5276cc81922a","Type":"ContainerStarted","Data":"6a60b9c9ac9837eb3664404556c4f435b7959f0547366b832c4d566f51b10871"}
Jul 16 22:24:04.940445 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.940341    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"
Jul 16 22:24:04.940747 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:04.940720    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blp4p\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:24:04.941695 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.941667    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg" event={"ID":"a9721a53-7880-477d-bf57-aeef00b239d0","Type":"ContainerStarted","Data":"12e95bf08d3be194fbf79a21f557ce34ec217ad5a41de47ef5706b248b105690"}
Jul 16 22:24:04.942524 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:04.942502    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:24:04.942811 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.942794    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:24:04.942874 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.942839    2556 scope.go:117] "RemoveContainer" containerID="a91fbece4c230e54cff83e07d519e65aa9da53dc212292d0d4c8a79a07201851"
Jul 16 22:24:04.944287 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.944268    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:24:04.952840 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.952814    2556 scope.go:117] "RemoveContainer" containerID="7d70cd5640bfe986862b6cf7e0fead25c065897574c4f63f7395076865357192"
Jul 16 22:24:04.963475 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.963435    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l" podStartSLOduration=14.963422006 podStartE2EDuration="14.963422006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:04.9619163 +0000 UTC m=+437.635175317" watchObservedRunningTime="2026-07-16 22:24:04.963422006 +0000 UTC m=+437.636681014"
Jul 16 22:24:04.963796 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:04.963771    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-89mwj\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-89mwj"
Jul 16 22:24:04.986563 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:04.986514    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z" podStartSLOduration=14.986502324 podStartE2EDuration="14.986502324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:04.986145603 +0000 UTC m=+437.659404629" watchObservedRunningTime="2026-07-16 22:24:04.986502324 +0000 UTC m=+437.659761333"
Jul 16 22:24:05.003071 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:05.003036    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blp4p\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blp4p"
Jul 16 22:24:05.003302 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:05.003263    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zprlg" podStartSLOduration=2.261631486 podStartE2EDuration="25.003249152s" totalImagesPullingTime="22.741617666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:05.001087474 +0000 UTC m=+437.674346510" watchObservedRunningTime="2026-07-16 22:24:05.003249152 +0000 UTC m=+437.676508159"
Jul 16 22:24:05.812840 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:05.812805    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a2f8803-edab-4237-9b9d-2c2ade6b179d" path="/var/lib/kubelet/pods/8a2f8803-edab-4237-9b9d-2c2ade6b179d/volumes"
Jul 16 22:24:05.813149 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:05.813136    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="deb88ce1-72fb-4c04-ad69-6ff563a2430c" path="/var/lib/kubelet/pods/deb88ce1-72fb-4c04-ad69-6ff563a2430c/volumes"
Jul 16 22:24:07.654537 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.654506    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"]
Jul 16 22:24:07.655004 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.654694    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l" podUID="9131e364-10d1-4712-8a8b-2354cb410a44" containerName="manager" containerID="cri-o://0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951" gracePeriod=10
Jul 16 22:24:07.922784 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.922763    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"]
Jul 16 22:24:07.941383 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.941353    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:07.944704 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.944681    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"]
Jul 16 22:24:07.947523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.947500    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:24:07.961166 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.961140    2556 generic.go:356] "Generic (PLEG): container finished" podID="9131e364-10d1-4712-8a8b-2354cb410a44" containerID="0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951" exitCode=0
Jul 16 22:24:07.961273 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.961181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l" event={"ID":"9131e364-10d1-4712-8a8b-2354cb410a44","Type":"ContainerDied","Data":"0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951"}
Jul 16 22:24:07.961273 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.961197    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"
Jul 16 22:24:07.961273 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.961239    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l" event={"ID":"9131e364-10d1-4712-8a8b-2354cb410a44","Type":"ContainerDied","Data":"4d7a57d7f13209dcbaeb8edfb5f0ec371afe988827b1784ff6af0dadea4b345a"}
Jul 16 22:24:07.961273 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.961262    2556 scope.go:117] "RemoveContainer" containerID="0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951"
Jul 16 22:24:07.971085 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.971067    2556 scope.go:117] "RemoveContainer" containerID="0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951"
Jul 16 22:24:07.971390 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:24:07.971370    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951\": container with ID starting with 0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951 not found: ID does not exist" containerID="0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951"
Jul 16 22:24:07.971450 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:07.971398    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951"} err="failed to get container status \"0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951\": rpc error: code = NotFound desc = could not find container \"0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951\": container with ID starting with 0abe8cb375a365e8e033bba3fa8e4760987ba9067ce1a2d6a48e4a7cc1182951 not found: ID does not exist"
Jul 16 22:24:08.011478 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.011438    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bk5k\" (UniqueName: \"kubernetes.io/projected/9131e364-10d1-4712-8a8b-2354cb410a44-kube-api-access-5bk5k\") pod \"9131e364-10d1-4712-8a8b-2354cb410a44\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") "
Jul 16 22:24:08.011631 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.011508    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9131e364-10d1-4712-8a8b-2354cb410a44-extensions-socket-volume\") pod \"9131e364-10d1-4712-8a8b-2354cb410a44\" (UID: \"9131e364-10d1-4712-8a8b-2354cb410a44\") "
Jul 16 22:24:08.011670 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.011647    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e3c7900-a966-4e00-a239-7c862445450b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mdg49\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.011701 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.011678    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7c6z\" (UniqueName: \"kubernetes.io/projected/0e3c7900-a966-4e00-a239-7c862445450b-kube-api-access-z7c6z\") pod \"kuadrant-operator-controller-manager-86878c769f-mdg49\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.011912 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.011889    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9131e364-10d1-4712-8a8b-2354cb410a44-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9131e364-10d1-4712-8a8b-2354cb410a44" (UID: "9131e364-10d1-4712-8a8b-2354cb410a44"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:24:08.013803 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.013778    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9131e364-10d1-4712-8a8b-2354cb410a44-kube-api-access-5bk5k" (OuterVolumeSpecName: "kube-api-access-5bk5k") pod "9131e364-10d1-4712-8a8b-2354cb410a44" (UID: "9131e364-10d1-4712-8a8b-2354cb410a44"). InnerVolumeSpecName "kube-api-access-5bk5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:24:08.112640 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.112601    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e3c7900-a966-4e00-a239-7c862445450b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mdg49\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.112785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.112650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7c6z\" (UniqueName: \"kubernetes.io/projected/0e3c7900-a966-4e00-a239-7c862445450b-kube-api-access-z7c6z\") pod \"kuadrant-operator-controller-manager-86878c769f-mdg49\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.112785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.112725    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bk5k\" (UniqueName: \"kubernetes.io/projected/9131e364-10d1-4712-8a8b-2354cb410a44-kube-api-access-5bk5k\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:24:08.112785 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.112741    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9131e364-10d1-4712-8a8b-2354cb410a44-extensions-socket-volume\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:24:08.112970 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.112951    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e3c7900-a966-4e00-a239-7c862445450b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mdg49\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.122678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.122655    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7c6z\" (UniqueName: \"kubernetes.io/projected/0e3c7900-a966-4e00-a239-7c862445450b-kube-api-access-z7c6z\") pod \"kuadrant-operator-controller-manager-86878c769f-mdg49\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.251990 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.251926    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.293685 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.293653    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"]
Jul 16 22:24:08.297987 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.297962    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hlb2l"]
Jul 16 22:24:08.358949 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.358917    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:08.432636 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.432613    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"]
Jul 16 22:24:08.435034 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:24:08.435010    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3c7900_a966_4e00_a239_7c862445450b.slice/crio-49ce5084b59770b6f3add5a5b96d7836207141f5766752966e82d0cd868d6ee4 WatchSource:0}: Error finding container 49ce5084b59770b6f3add5a5b96d7836207141f5766752966e82d0cd868d6ee4: Status 404 returned error can't find the container with id 49ce5084b59770b6f3add5a5b96d7836207141f5766752966e82d0cd868d6ee4
Jul 16 22:24:08.527075 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.527042    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:08.968983 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.968944    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49" event={"ID":"0e3c7900-a966-4e00-a239-7c862445450b","Type":"ContainerStarted","Data":"473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c"}
Jul 16 22:24:08.969494 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.968997    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49" event={"ID":"0e3c7900-a966-4e00-a239-7c862445450b","Type":"ContainerStarted","Data":"49ce5084b59770b6f3add5a5b96d7836207141f5766752966e82d0cd868d6ee4"}
Jul 16 22:24:08.969494 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.969027    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:08.990256 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:08.990202    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49" podStartSLOduration=1.9901818470000001 podStartE2EDuration="1.990181847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:08.987849379 +0000 UTC m=+441.661108385" watchObservedRunningTime="2026-07-16 22:24:08.990181847 +0000 UTC m=+441.663440858"
Jul 16 22:24:09.812656 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:09.812621    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9131e364-10d1-4712-8a8b-2354cb410a44" path="/var/lib/kubelet/pods/9131e364-10d1-4712-8a8b-2354cb410a44/volumes"
Jul 16 22:24:10.967212 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:10.967157    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:11.068502 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:11.068456    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:14.152711 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:14.152669    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:14.260137 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:14.260067    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:15.953560 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:15.953528    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9488z"
Jul 16 22:24:17.855971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:17.855904    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:17.960630 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:17.960596    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:19.976612 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:19.976582    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:24:24.079994 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:24.079947    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:24.192717 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:24.192668    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:27.359094 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:27.359056    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:27.454206 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:27.454167    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:27.958035 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:27.957990    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:28.020918 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:28.020881    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:38.218484 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:38.218448    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:38.331188 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:38.331145    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:38.387480 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:38.387446    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:39.087290 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.087246    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:39.188856 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.188808    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:39.203521 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.203490    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jtnd"]
Jul 16 22:24:39.203927 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.203909    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9131e364-10d1-4712-8a8b-2354cb410a44" containerName="manager"
Jul 16 22:24:39.204011 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.203929    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9131e364-10d1-4712-8a8b-2354cb410a44" containerName="manager"
Jul 16 22:24:39.204069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.204012    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9131e364-10d1-4712-8a8b-2354cb410a44" containerName="manager"
Jul 16 22:24:39.206096 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.206074    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jtnd"
Jul 16 22:24:39.209416 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.209391    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qvr2t\""
Jul 16 22:24:39.213321 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.213292    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jtnd"]
Jul 16 22:24:39.292385 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.292352    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtrhl\" (UniqueName: \"kubernetes.io/projected/4706d4f5-c066-454d-ab8b-9d2fd5596574-kube-api-access-wtrhl\") pod \"authorino-75d556bd77-9jtnd\" (UID: \"4706d4f5-c066-454d-ab8b-9d2fd5596574\") " pod="kuadrant-system/authorino-75d556bd77-9jtnd"
Jul 16 22:24:39.393406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.393324    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtrhl\" (UniqueName: \"kubernetes.io/projected/4706d4f5-c066-454d-ab8b-9d2fd5596574-kube-api-access-wtrhl\") pod \"authorino-75d556bd77-9jtnd\" (UID: \"4706d4f5-c066-454d-ab8b-9d2fd5596574\") " pod="kuadrant-system/authorino-75d556bd77-9jtnd"
Jul 16 22:24:39.402423 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.402401    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtrhl\" (UniqueName: \"kubernetes.io/projected/4706d4f5-c066-454d-ab8b-9d2fd5596574-kube-api-access-wtrhl\") pod \"authorino-75d556bd77-9jtnd\" (UID: \"4706d4f5-c066-454d-ab8b-9d2fd5596574\") " pod="kuadrant-system/authorino-75d556bd77-9jtnd"
Jul 16 22:24:39.516700 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.516668    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jtnd"
Jul 16 22:24:39.646831 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:39.646741    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jtnd"]
Jul 16 22:24:39.651188 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:24:39.651101    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4706d4f5_c066_454d_ab8b_9d2fd5596574.slice/crio-1aa4bd95898440e18e6058ce62528673def03189af435fa85014373182f12b06 WatchSource:0}: Error finding container 1aa4bd95898440e18e6058ce62528673def03189af435fa85014373182f12b06: Status 404 returned error can't find the container with id 1aa4bd95898440e18e6058ce62528673def03189af435fa85014373182f12b06
Jul 16 22:24:40.132958 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:40.132919    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jtnd" event={"ID":"4706d4f5-c066-454d-ab8b-9d2fd5596574","Type":"ContainerStarted","Data":"1aa4bd95898440e18e6058ce62528673def03189af435fa85014373182f12b06"}
Jul 16 22:24:44.154580 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:44.154537    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jtnd" event={"ID":"4706d4f5-c066-454d-ab8b-9d2fd5596574","Type":"ContainerStarted","Data":"2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6"}
Jul 16 22:24:44.169385 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:44.169338    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9jtnd" podStartSLOduration=1.660241526 podStartE2EDuration="5.169325753s" totalImagesPullingTime="3.509084227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:24:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:44.168381576 +0000 UTC m=+476.841640585" watchObservedRunningTime="2026-07-16 22:24:44.169325753 +0000 UTC m=+476.842584761"
Jul 16 22:24:57.608406 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:57.608371    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:57.950925 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:57.950844    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:58.051270 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:24:58.051239    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:09.824748 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:09.824707    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:10.304309 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.304272    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b6f7bcf75-gttk9"]
Jul 16 22:25:10.307878 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.307857    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9"
Jul 16 22:25:10.315260 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.315232    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b6f7bcf75-gttk9"]
Jul 16 22:25:10.349207 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.349172    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj89k\" (UniqueName: \"kubernetes.io/projected/7b4afeee-8e17-4945-a33c-89ba63ecb7d8-kube-api-access-fj89k\") pod \"authorino-7b6f7bcf75-gttk9\" (UID: \"7b4afeee-8e17-4945-a33c-89ba63ecb7d8\") " pod="kuadrant-system/authorino-7b6f7bcf75-gttk9"
Jul 16 22:25:10.390325 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.390287    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:10.450433 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.450401    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj89k\" (UniqueName: \"kubernetes.io/projected/7b4afeee-8e17-4945-a33c-89ba63ecb7d8-kube-api-access-fj89k\") pod \"authorino-7b6f7bcf75-gttk9\" (UID: \"7b4afeee-8e17-4945-a33c-89ba63ecb7d8\") " pod="kuadrant-system/authorino-7b6f7bcf75-gttk9"
Jul 16 22:25:10.458438 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.458416    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj89k\" (UniqueName: \"kubernetes.io/projected/7b4afeee-8e17-4945-a33c-89ba63ecb7d8-kube-api-access-fj89k\") pod \"authorino-7b6f7bcf75-gttk9\" (UID: \"7b4afeee-8e17-4945-a33c-89ba63ecb7d8\") " pod="kuadrant-system/authorino-7b6f7bcf75-gttk9"
Jul 16 22:25:10.568024 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.567955    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b6f7bcf75-gttk9"]
Jul 16 22:25:10.568221 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.568205    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9"
Jul 16 22:25:10.717720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:10.717693    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b6f7bcf75-gttk9"]
Jul 16 22:25:10.720610 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:25:10.720579    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b4afeee_8e17_4945_a33c_89ba63ecb7d8.slice/crio-a230a5df5555a99c978c0144ec7bcccc11ec981f26fe513a73c4343d5eeff9c8 WatchSource:0}: Error finding container a230a5df5555a99c978c0144ec7bcccc11ec981f26fe513a73c4343d5eeff9c8: Status 404 returned error can't find the container with id a230a5df5555a99c978c0144ec7bcccc11ec981f26fe513a73c4343d5eeff9c8
Jul 16 22:25:11.281008 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.280975    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9" event={"ID":"7b4afeee-8e17-4945-a33c-89ba63ecb7d8","Type":"ContainerStarted","Data":"2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046"}
Jul 16 22:25:11.281461 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.281022    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9" event={"ID":"7b4afeee-8e17-4945-a33c-89ba63ecb7d8","Type":"ContainerStarted","Data":"a230a5df5555a99c978c0144ec7bcccc11ec981f26fe513a73c4343d5eeff9c8"}
Jul 16 22:25:11.281461 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.281034    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9" podUID="7b4afeee-8e17-4945-a33c-89ba63ecb7d8" containerName="authorino" containerID="cri-o://2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046" gracePeriod=30
Jul 16 22:25:11.296069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.296019    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9" podStartSLOduration=0.970967296 podStartE2EDuration="1.296003895s" totalImagesPullingTime="325.036599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:25:11.2947846 +0000 UTC m=+503.968043621" watchObservedRunningTime="2026-07-16 22:25:11.296003895 +0000 UTC m=+503.969262902"
Jul 16 22:25:11.529752 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.529726    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9"
Jul 16 22:25:11.560234 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.560200    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj89k\" (UniqueName: \"kubernetes.io/projected/7b4afeee-8e17-4945-a33c-89ba63ecb7d8-kube-api-access-fj89k\") pod \"7b4afeee-8e17-4945-a33c-89ba63ecb7d8\" (UID: \"7b4afeee-8e17-4945-a33c-89ba63ecb7d8\") "
Jul 16 22:25:11.562462 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.562425    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4afeee-8e17-4945-a33c-89ba63ecb7d8-kube-api-access-fj89k" (OuterVolumeSpecName: "kube-api-access-fj89k") pod "7b4afeee-8e17-4945-a33c-89ba63ecb7d8" (UID: "7b4afeee-8e17-4945-a33c-89ba63ecb7d8"). InnerVolumeSpecName "kube-api-access-fj89k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:25:11.660881 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:11.660797    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj89k\" (UniqueName: \"kubernetes.io/projected/7b4afeee-8e17-4945-a33c-89ba63ecb7d8-kube-api-access-fj89k\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:25:12.286801 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.286764    2556 generic.go:356] "Generic (PLEG): container finished" podID="7b4afeee-8e17-4945-a33c-89ba63ecb7d8" containerID="2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046" exitCode=0
Jul 16 22:25:12.287286 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.286808    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9"
Jul 16 22:25:12.287286 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.286809    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9" event={"ID":"7b4afeee-8e17-4945-a33c-89ba63ecb7d8","Type":"ContainerDied","Data":"2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046"}
Jul 16 22:25:12.287286 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.286916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b6f7bcf75-gttk9" event={"ID":"7b4afeee-8e17-4945-a33c-89ba63ecb7d8","Type":"ContainerDied","Data":"a230a5df5555a99c978c0144ec7bcccc11ec981f26fe513a73c4343d5eeff9c8"}
Jul 16 22:25:12.287286 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.286943    2556 scope.go:117] "RemoveContainer" containerID="2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046"
Jul 16 22:25:12.295934 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.295916    2556 scope.go:117] "RemoveContainer" containerID="2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046"
Jul 16 22:25:12.296238 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:25:12.296215    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046\": container with ID starting with 2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046 not found: ID does not exist" containerID="2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046"
Jul 16 22:25:12.296341 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.296249    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046"} err="failed to get container status \"2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046\": rpc error: code = NotFound desc = could not find container \"2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046\": container with ID starting with 2ad30e05f7d48285040a1e223a3dd7cdb7f523b362aff6bc5ec9e7a23afb4046 not found: ID does not exist"
Jul 16 22:25:12.301667 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.301634    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b6f7bcf75-gttk9"]
Jul 16 22:25:12.305869 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.305846    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b6f7bcf75-gttk9"]
Jul 16 22:25:12.552738 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.552644    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jtnd"]
Jul 16 22:25:12.554980 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.553329    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9jtnd" podUID="4706d4f5-c066-454d-ab8b-9d2fd5596574" containerName="authorino" containerID="cri-o://2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6" gracePeriod=30
Jul 16 22:25:12.803357 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.803298    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jtnd"
Jul 16 22:25:12.869951 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.869916    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtrhl\" (UniqueName: \"kubernetes.io/projected/4706d4f5-c066-454d-ab8b-9d2fd5596574-kube-api-access-wtrhl\") pod \"4706d4f5-c066-454d-ab8b-9d2fd5596574\" (UID: \"4706d4f5-c066-454d-ab8b-9d2fd5596574\") "
Jul 16 22:25:12.872191 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.872161    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4706d4f5-c066-454d-ab8b-9d2fd5596574-kube-api-access-wtrhl" (OuterVolumeSpecName: "kube-api-access-wtrhl") pod "4706d4f5-c066-454d-ab8b-9d2fd5596574" (UID: "4706d4f5-c066-454d-ab8b-9d2fd5596574"). InnerVolumeSpecName "kube-api-access-wtrhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:25:12.970561 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:12.970535    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtrhl\" (UniqueName: \"kubernetes.io/projected/4706d4f5-c066-454d-ab8b-9d2fd5596574-kube-api-access-wtrhl\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:25:13.294823 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.294787    2556 generic.go:356] "Generic (PLEG): container finished" podID="4706d4f5-c066-454d-ab8b-9d2fd5596574" containerID="2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6" exitCode=0
Jul 16 22:25:13.294823 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.294823    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9jtnd"
Jul 16 22:25:13.295357 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.294882    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jtnd" event={"ID":"4706d4f5-c066-454d-ab8b-9d2fd5596574","Type":"ContainerDied","Data":"2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6"}
Jul 16 22:25:13.295357 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.294926    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9jtnd" event={"ID":"4706d4f5-c066-454d-ab8b-9d2fd5596574","Type":"ContainerDied","Data":"1aa4bd95898440e18e6058ce62528673def03189af435fa85014373182f12b06"}
Jul 16 22:25:13.295357 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.294944    2556 scope.go:117] "RemoveContainer" containerID="2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6"
Jul 16 22:25:13.303983 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.303967    2556 scope.go:117] "RemoveContainer" containerID="2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6"
Jul 16 22:25:13.304272 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:25:13.304253    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6\": container with ID starting with 2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6 not found: ID does not exist" containerID="2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6"
Jul 16 22:25:13.304319 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.304283    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6"} err="failed to get container status \"2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6\": rpc error: code = NotFound desc = could not find container \"2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6\": container with ID starting with 2b0e9b3283fca0119e2a15e6c89e9583f47f8f440a23befbbf33387460d9aac6 not found: ID does not exist"
Jul 16 22:25:13.315665 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.315643    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jtnd"]
Jul 16 22:25:13.319489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.319469    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9jtnd"]
Jul 16 22:25:13.811326 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.811298    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4706d4f5-c066-454d-ab8b-9d2fd5596574" path="/var/lib/kubelet/pods/4706d4f5-c066-454d-ab8b-9d2fd5596574/volumes"
Jul 16 22:25:13.811604 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:13.811591    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b4afeee-8e17-4945-a33c-89ba63ecb7d8" path="/var/lib/kubelet/pods/7b4afeee-8e17-4945-a33c-89ba63ecb7d8/volumes"
Jul 16 22:25:17.138527 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.138490    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75646995f5-f6n55"]
Jul 16 22:25:17.138896 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.138844    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4706d4f5-c066-454d-ab8b-9d2fd5596574" containerName="authorino"
Jul 16 22:25:17.138896 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.138857    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4706d4f5-c066-454d-ab8b-9d2fd5596574" containerName="authorino"
Jul 16 22:25:17.138896 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.138876    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b4afeee-8e17-4945-a33c-89ba63ecb7d8" containerName="authorino"
Jul 16 22:25:17.138896 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.138883    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4afeee-8e17-4945-a33c-89ba63ecb7d8" containerName="authorino"
Jul 16 22:25:17.139032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.138946    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4706d4f5-c066-454d-ab8b-9d2fd5596574" containerName="authorino"
Jul 16 22:25:17.139032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.138955    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b4afeee-8e17-4945-a33c-89ba63ecb7d8" containerName="authorino"
Jul 16 22:25:17.187398 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.187365    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-f6n55"]
Jul 16 22:25:17.187584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.187505    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.190269 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.190246    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 22:25:17.190409 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.190326    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mk6jb\""
Jul 16 22:25:17.192058 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.192034    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 22:25:17.310586 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.310553    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f086466b-30d4-4d4e-9695-3da330c3534c-webhook-cert\") pod \"maas-controller-75646995f5-f6n55\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") " pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.310751 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.310716    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf6dh\" (UniqueName: \"kubernetes.io/projected/f086466b-30d4-4d4e-9695-3da330c3534c-kube-api-access-cf6dh\") pod \"maas-controller-75646995f5-f6n55\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") " pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.320162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.320130    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79cbb885f5-sjhb7"]
Jul 16 22:25:17.352689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.352648    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-sjhb7"]
Jul 16 22:25:17.352843 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.352691    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.408808 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.408722    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:17.411853 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.411826    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf6dh\" (UniqueName: \"kubernetes.io/projected/f086466b-30d4-4d4e-9695-3da330c3534c-kube-api-access-cf6dh\") pod \"maas-controller-75646995f5-f6n55\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") " pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.411974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.411867    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f086466b-30d4-4d4e-9695-3da330c3534c-webhook-cert\") pod \"maas-controller-75646995f5-f6n55\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") " pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.411974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.411916    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hknk\" (UniqueName: \"kubernetes.io/projected/553bb3ce-7944-4fd1-ada2-5c26cb435037-kube-api-access-8hknk\") pod \"maas-controller-79cbb885f5-sjhb7\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") " pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.412093 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.412077    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/553bb3ce-7944-4fd1-ada2-5c26cb435037-webhook-cert\") pod \"maas-controller-79cbb885f5-sjhb7\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") " pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.414536 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.414514    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f086466b-30d4-4d4e-9695-3da330c3534c-webhook-cert\") pod \"maas-controller-75646995f5-f6n55\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") " pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.420332 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.420308    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf6dh\" (UniqueName: \"kubernetes.io/projected/f086466b-30d4-4d4e-9695-3da330c3534c-kube-api-access-cf6dh\") pod \"maas-controller-75646995f5-f6n55\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") " pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.466514 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.466459    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:17.498433 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.498407    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:17.512798 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.512748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/553bb3ce-7944-4fd1-ada2-5c26cb435037-webhook-cert\") pod \"maas-controller-79cbb885f5-sjhb7\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") " pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.512929 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.512835    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hknk\" (UniqueName: \"kubernetes.io/projected/553bb3ce-7944-4fd1-ada2-5c26cb435037-kube-api-access-8hknk\") pod \"maas-controller-79cbb885f5-sjhb7\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") " pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.515556 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.515532    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/553bb3ce-7944-4fd1-ada2-5c26cb435037-webhook-cert\") pod \"maas-controller-79cbb885f5-sjhb7\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") " pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.522355 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.522332    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hknk\" (UniqueName: \"kubernetes.io/projected/553bb3ce-7944-4fd1-ada2-5c26cb435037-kube-api-access-8hknk\") pod \"maas-controller-79cbb885f5-sjhb7\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") " pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.631787 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.631760    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-f6n55"]
Jul 16 22:25:17.634406 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:25:17.634379    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf086466b_30d4_4d4e_9695_3da330c3534c.slice/crio-092a74cb8511f1124f5153ad226278b56223c0379bdef85e262d7a9e6dbb684f WatchSource:0}: Error finding container 092a74cb8511f1124f5153ad226278b56223c0379bdef85e262d7a9e6dbb684f: Status 404 returned error can't find the container with id 092a74cb8511f1124f5153ad226278b56223c0379bdef85e262d7a9e6dbb684f
Jul 16 22:25:17.663147 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.663053    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:17.797086 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:17.797065    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-sjhb7"]
Jul 16 22:25:17.799263 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:25:17.799235    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553bb3ce_7944_4fd1_ada2_5c26cb435037.slice/crio-04d7ba8b8ef0f4564dc584589b92bcbfb98667bbf42e75f01814159b74167329 WatchSource:0}: Error finding container 04d7ba8b8ef0f4564dc584589b92bcbfb98667bbf42e75f01814159b74167329: Status 404 returned error can't find the container with id 04d7ba8b8ef0f4564dc584589b92bcbfb98667bbf42e75f01814159b74167329
Jul 16 22:25:18.319749 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:18.319705    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-sjhb7" event={"ID":"553bb3ce-7944-4fd1-ada2-5c26cb435037","Type":"ContainerStarted","Data":"04d7ba8b8ef0f4564dc584589b92bcbfb98667bbf42e75f01814159b74167329"}
Jul 16 22:25:18.320968 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:18.320947    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-f6n55" event={"ID":"f086466b-30d4-4d4e-9695-3da330c3534c","Type":"ContainerStarted","Data":"092a74cb8511f1124f5153ad226278b56223c0379bdef85e262d7a9e6dbb684f"}
Jul 16 22:25:20.743300 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:20.743259    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:20.844153 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:20.844048    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:21.340669 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:21.340628    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-sjhb7" event={"ID":"553bb3ce-7944-4fd1-ada2-5c26cb435037","Type":"ContainerStarted","Data":"5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe"}
Jul 16 22:25:21.340862 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:21.340803    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:21.342128 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:21.342092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-f6n55" event={"ID":"f086466b-30d4-4d4e-9695-3da330c3534c","Type":"ContainerStarted","Data":"e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621"}
Jul 16 22:25:21.342293 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:21.342152    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:21.359701 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:21.359657    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79cbb885f5-sjhb7" podStartSLOduration=1.5976457389999998 podStartE2EDuration="4.35964518s" totalImagesPullingTime="2.761999441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:25:21.357593464 +0000 UTC m=+514.030852489" watchObservedRunningTime="2026-07-16 22:25:21.35964518 +0000 UTC m=+514.032904187"
Jul 16 22:25:21.376651 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:21.376599    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75646995f5-f6n55" podStartSLOduration=1.460494073 podStartE2EDuration="4.376582863s" totalImagesPullingTime="2.91608879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:25:21.375156697 +0000 UTC m=+514.048415706" watchObservedRunningTime="2026-07-16 22:25:21.376582863 +0000 UTC m=+514.049841870"
Jul 16 22:25:32.349099 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.349064    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:25:32.349584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.349169    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:32.412659 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.412622    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-f6n55"]
Jul 16 22:25:32.412839 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.412800    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75646995f5-f6n55" podUID="f086466b-30d4-4d4e-9695-3da330c3534c" containerName="manager" containerID="cri-o://e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621" gracePeriod=30
Jul 16 22:25:32.656207 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.656185    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:32.754043 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.754006    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf6dh\" (UniqueName: \"kubernetes.io/projected/f086466b-30d4-4d4e-9695-3da330c3534c-kube-api-access-cf6dh\") pod \"f086466b-30d4-4d4e-9695-3da330c3534c\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") "
Jul 16 22:25:32.754245 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.754059    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f086466b-30d4-4d4e-9695-3da330c3534c-webhook-cert\") pod \"f086466b-30d4-4d4e-9695-3da330c3534c\" (UID: \"f086466b-30d4-4d4e-9695-3da330c3534c\") "
Jul 16 22:25:32.756437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.756406    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f086466b-30d4-4d4e-9695-3da330c3534c-kube-api-access-cf6dh" (OuterVolumeSpecName: "kube-api-access-cf6dh") pod "f086466b-30d4-4d4e-9695-3da330c3534c" (UID: "f086466b-30d4-4d4e-9695-3da330c3534c"). InnerVolumeSpecName "kube-api-access-cf6dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:25:32.756437 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.756426    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f086466b-30d4-4d4e-9695-3da330c3534c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f086466b-30d4-4d4e-9695-3da330c3534c" (UID: "f086466b-30d4-4d4e-9695-3da330c3534c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:25:32.854905 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.854871    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f086466b-30d4-4d4e-9695-3da330c3534c-webhook-cert\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:25:32.854905 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:32.854901    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf6dh\" (UniqueName: \"kubernetes.io/projected/f086466b-30d4-4d4e-9695-3da330c3534c-kube-api-access-cf6dh\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:25:33.217627 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.217593    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:33.400873 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.400837    2556 generic.go:356] "Generic (PLEG): container finished" podID="f086466b-30d4-4d4e-9695-3da330c3534c" containerID="e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621" exitCode=0
Jul 16 22:25:33.401331 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.400882    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-f6n55"
Jul 16 22:25:33.401331 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.400925    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-f6n55" event={"ID":"f086466b-30d4-4d4e-9695-3da330c3534c","Type":"ContainerDied","Data":"e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621"}
Jul 16 22:25:33.401331 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.400978    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-f6n55" event={"ID":"f086466b-30d4-4d4e-9695-3da330c3534c","Type":"ContainerDied","Data":"092a74cb8511f1124f5153ad226278b56223c0379bdef85e262d7a9e6dbb684f"}
Jul 16 22:25:33.401331 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.400999    2556 scope.go:117] "RemoveContainer" containerID="e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621"
Jul 16 22:25:33.409851 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.409833    2556 scope.go:117] "RemoveContainer" containerID="e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621"
Jul 16 22:25:33.410097 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:25:33.410079    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621\": container with ID starting with e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621 not found: ID does not exist" containerID="e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621"
Jul 16 22:25:33.410184 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.410121    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621"} err="failed to get container status \"e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621\": rpc error: code = NotFound desc = could not find container \"e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621\": container with ID starting with e59e5ab47f3798e7899d269c0310102fc8a231b04ac4e07db0169c03fd17c621 not found: ID does not exist"
Jul 16 22:25:33.421893 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.421870    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-f6n55"]
Jul 16 22:25:33.428062 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.428038    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75646995f5-f6n55"]
Jul 16 22:25:33.811587 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:33.811554    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f086466b-30d4-4d4e-9695-3da330c3534c" path="/var/lib/kubelet/pods/f086466b-30d4-4d4e-9695-3da330c3534c/volumes"
Jul 16 22:25:52.248662 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.248623    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:52.343496 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.343451    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:52.797669 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.797634    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"]
Jul 16 22:25:52.798055 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.798040    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f086466b-30d4-4d4e-9695-3da330c3534c" containerName="manager"
Jul 16 22:25:52.798131 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.798058    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f086466b-30d4-4d4e-9695-3da330c3534c" containerName="manager"
Jul 16 22:25:52.798171 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.798143    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f086466b-30d4-4d4e-9695-3da330c3534c" containerName="manager"
Jul 16 22:25:52.802389 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.802372    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.804952 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.804929    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 16 22:25:52.805065 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.805009    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:25:52.805065 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.805009    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 22:25:52.805936 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.805907    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:52.811868 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.811848    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"]
Jul 16 22:25:52.835523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.835497    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea89920-e85b-489a-b481-b82c342f57af-openshift-service-ca\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.835691 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.835535    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh98j\" (UniqueName: \"kubernetes.io/projected/9ea89920-e85b-489a-b481-b82c342f57af-kube-api-access-jh98j\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.835691 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.835560    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea89920-e85b-489a-b481-b82c342f57af-maas-api-tls\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.937031 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.936994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea89920-e85b-489a-b481-b82c342f57af-openshift-service-ca\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.937233 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.937044    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh98j\" (UniqueName: \"kubernetes.io/projected/9ea89920-e85b-489a-b481-b82c342f57af-kube-api-access-jh98j\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.937233 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.937082    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea89920-e85b-489a-b481-b82c342f57af-maas-api-tls\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.937330 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:25:52.937248    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 16 22:25:52.937330 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:25:52.937320    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea89920-e85b-489a-b481-b82c342f57af-maas-api-tls podName:9ea89920-e85b-489a-b481-b82c342f57af nodeName:}" failed. No retries permitted until 2026-07-16 22:25:53.437299958 +0000 UTC m=+546.110558950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ea89920-e85b-489a-b481-b82c342f57af-maas-api-tls") pod "maas-api-7954f7d7ff-rj66x" (UID: "9ea89920-e85b-489a-b481-b82c342f57af") : secret "maas-api-serving-cert" not found
Jul 16 22:25:52.937675 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.937657    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea89920-e85b-489a-b481-b82c342f57af-openshift-service-ca\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:52.948046 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:52.948017    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh98j\" (UniqueName: \"kubernetes.io/projected/9ea89920-e85b-489a-b481-b82c342f57af-kube-api-access-jh98j\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:53.132106 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.132023    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:53.389199 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.389028    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:53.442238 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.442204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea89920-e85b-489a-b481-b82c342f57af-maas-api-tls\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:53.444963 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.444939    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea89920-e85b-489a-b481-b82c342f57af-maas-api-tls\") pod \"maas-api-7954f7d7ff-rj66x\" (UID: \"9ea89920-e85b-489a-b481-b82c342f57af\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:53.479561 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.479531    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:53.483882 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.483861    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"]
Jul 16 22:25:53.487492 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.487467    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.489675 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.489636    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 16 22:25:53.489780 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.489693    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-k7xts\""
Jul 16 22:25:53.499445 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.499420    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"]
Jul 16 22:25:53.543592 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.543559    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skzk6\" (UniqueName: \"kubernetes.io/projected/13985c55-246e-4408-a04b-e7b95d6e72f3-kube-api-access-skzk6\") pod \"payload-pre-processing-7677b774bb-hd7z8\" (UID: \"13985c55-246e-4408-a04b-e7b95d6e72f3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.543752 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.543610    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13985c55-246e-4408-a04b-e7b95d6e72f3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-hd7z8\" (UID: \"13985c55-246e-4408-a04b-e7b95d6e72f3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.644747 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.644671    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skzk6\" (UniqueName: \"kubernetes.io/projected/13985c55-246e-4408-a04b-e7b95d6e72f3-kube-api-access-skzk6\") pod \"payload-pre-processing-7677b774bb-hd7z8\" (UID: \"13985c55-246e-4408-a04b-e7b95d6e72f3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.644747 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.644717    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13985c55-246e-4408-a04b-e7b95d6e72f3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-hd7z8\" (UID: \"13985c55-246e-4408-a04b-e7b95d6e72f3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.645489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.645468    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13985c55-246e-4408-a04b-e7b95d6e72f3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-hd7z8\" (UID: \"13985c55-246e-4408-a04b-e7b95d6e72f3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.655988 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.655967    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skzk6\" (UniqueName: \"kubernetes.io/projected/13985c55-246e-4408-a04b-e7b95d6e72f3-kube-api-access-skzk6\") pod \"payload-pre-processing-7677b774bb-hd7z8\" (UID: \"13985c55-246e-4408-a04b-e7b95d6e72f3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.715218 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.715179    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:53.799530 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.799499    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:53.870532 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.870377    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"]
Jul 16 22:25:53.875605 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:25:53.875578    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea89920_e85b_489a_b481_b82c342f57af.slice/crio-5cbadaff4fd4287e6b2bd1d2c6051ff2eb34c4a834a4c0750cc673a29f647518 WatchSource:0}: Error finding container 5cbadaff4fd4287e6b2bd1d2c6051ff2eb34c4a834a4c0750cc673a29f647518: Status 404 returned error can't find the container with id 5cbadaff4fd4287e6b2bd1d2c6051ff2eb34c4a834a4c0750cc673a29f647518
Jul 16 22:25:53.953044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:53.953006    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"]
Jul 16 22:25:53.957238 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:25:53.957210    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13985c55_246e_4408_a04b_e7b95d6e72f3.slice/crio-e0cba64731cf9dc238c976ad209cd83fd61dc554850d406c9270a747729cc28e WatchSource:0}: Error finding container e0cba64731cf9dc238c976ad209cd83fd61dc554850d406c9270a747729cc28e: Status 404 returned error can't find the container with id e0cba64731cf9dc238c976ad209cd83fd61dc554850d406c9270a747729cc28e
Jul 16 22:25:54.499892 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:54.499853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x" event={"ID":"9ea89920-e85b-489a-b481-b82c342f57af","Type":"ContainerStarted","Data":"5cbadaff4fd4287e6b2bd1d2c6051ff2eb34c4a834a4c0750cc673a29f647518"}
Jul 16 22:25:54.501073 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:54.501046    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8" event={"ID":"13985c55-246e-4408-a04b-e7b95d6e72f3","Type":"ContainerStarted","Data":"e0cba64731cf9dc238c976ad209cd83fd61dc554850d406c9270a747729cc28e"}
Jul 16 22:25:57.519617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:57.519585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x" event={"ID":"9ea89920-e85b-489a-b481-b82c342f57af","Type":"ContainerStarted","Data":"33d2a88c03df5db9ed822b06abb1f299be67b116d1e99b200c9e328948ab6773"}
Jul 16 22:25:57.520086 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:57.519708    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:25:57.521287 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:57.521264    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8" event={"ID":"13985c55-246e-4408-a04b-e7b95d6e72f3","Type":"ContainerStarted","Data":"d601a11b25f7d386e08328b13019b3e1f21d62dd4c09fcb6d64fdef48739bc12"}
Jul 16 22:25:57.521411 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:57.521335    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:25:57.540150 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:57.540085    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x" podStartSLOduration=2.727724258 podStartE2EDuration="5.540069978s" totalImagesPullingTime="2.81234572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:25:57.538631694 +0000 UTC m=+550.211890702" watchObservedRunningTime="2026-07-16 22:25:57.540069978 +0000 UTC m=+550.213328985"
Jul 16 22:25:57.555730 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:25:57.555688    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8" podStartSLOduration=1.825376691 podStartE2EDuration="4.55567294s" totalImagesPullingTime="2.730296249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:25:57.554066781 +0000 UTC m=+550.227325789" watchObservedRunningTime="2026-07-16 22:25:57.55567294 +0000 UTC m=+550.228931948"
Jul 16 22:26:03.497869 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:03.497832    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:03.531885 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:03.531859    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rj66x"
Jul 16 22:26:03.558091 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:03.558061    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:04.512507 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:04.512461    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:04.612014 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:04.611982    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:08.528584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:08.528556    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-hd7z8"
Jul 16 22:26:10.108300 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:10.108267    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:20.700711 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:20.700666    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:25.119055 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.119020    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"]
Jul 16 22:26:25.121760 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.121740    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.127044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.127019    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 16 22:26:25.127842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.127814    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 22:26:25.127842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.127830    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zttcm\""
Jul 16 22:26:25.128034 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.127855    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 22:26:25.147274 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.147244    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"]
Jul 16 22:26:25.167570 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.167536    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:25.227107 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.227071    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ccfa151-01ea-4f55-9e6b-43253067498d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.227107 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.227135    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9pvj\" (UniqueName: \"kubernetes.io/projected/5ccfa151-01ea-4f55-9e6b-43253067498d-kube-api-access-m9pvj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.227364 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.227209    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.227364 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.227248    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.227364 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.227277    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.227364 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.227334    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.328877 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.328826    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ccfa151-01ea-4f55-9e6b-43253067498d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.328890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9pvj\" (UniqueName: \"kubernetes.io/projected/5ccfa151-01ea-4f55-9e6b-43253067498d-kube-api-access-m9pvj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.328947    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.328972    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.328994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.329046    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329487 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.329460    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.329484    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.329576 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.329534    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.331513 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.331486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ccfa151-01ea-4f55-9e6b-43253067498d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.331612 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.331568    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ccfa151-01ea-4f55-9e6b-43253067498d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.337540 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.337517    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9pvj\" (UniqueName: \"kubernetes.io/projected/5ccfa151-01ea-4f55-9e6b-43253067498d-kube-api-access-m9pvj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tdq9s\" (UID: \"5ccfa151-01ea-4f55-9e6b-43253067498d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.432338 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.432246    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:25.577348 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.577324    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"]
Jul 16 22:26:25.579198 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:26:25.579157    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccfa151_01ea_4f55_9e6b_43253067498d.slice/crio-50e27ef1ebd54aaa38104bb239c31ed13ea3a40f953318bbacb08ed62464007e WatchSource:0}: Error finding container 50e27ef1ebd54aaa38104bb239c31ed13ea3a40f953318bbacb08ed62464007e: Status 404 returned error can't find the container with id 50e27ef1ebd54aaa38104bb239c31ed13ea3a40f953318bbacb08ed62464007e
Jul 16 22:26:25.652239 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:25.652203    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s" event={"ID":"5ccfa151-01ea-4f55-9e6b-43253067498d","Type":"ContainerStarted","Data":"50e27ef1ebd54aaa38104bb239c31ed13ea3a40f953318bbacb08ed62464007e"}
Jul 16 22:26:27.663936 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:27.663897    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s" event={"ID":"5ccfa151-01ea-4f55-9e6b-43253067498d","Type":"ContainerStarted","Data":"6ca5db3ca579678f59a4643bae7f5d59781037a2af9c33c4c76496ccd553ccb3"}
Jul 16 22:26:27.690336 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:27.690283    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s" podStartSLOduration=1.6824533000000002 podStartE2EDuration="2.690268602s" totalImagesPullingTime="1.007815302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:26:27.688610919 +0000 UTC m=+580.361869927" watchObservedRunningTime="2026-07-16 22:26:27.690268602 +0000 UTC m=+580.363527608"
Jul 16 22:26:30.857100 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.857064    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"]
Jul 16 22:26:30.864288 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.864266    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.867497 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.867474    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 16 22:26:30.873447 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.873413    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.873586 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.873465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.873586 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.873527    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.873692 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.873609    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.873692 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.873645    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99gdc\" (UniqueName: \"kubernetes.io/projected/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-kube-api-access-99gdc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.873692 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.873679    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.873908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.873887    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"]
Jul 16 22:26:30.895310 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.895278    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:30.974076 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974076 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974079    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974320 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974138    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974320 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974176    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974320 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974199    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99gdc\" (UniqueName: \"kubernetes.io/projected/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-kube-api-access-99gdc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974320 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974222    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974634 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974599    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974634 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.974838 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.974656    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.976597 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.976576    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.976838 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.976823    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:30.981870 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:30.981849    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99gdc\" (UniqueName: \"kubernetes.io/projected/c25fde28-2e3d-4cf3-8a43-86f2c09a05dd-kube-api-access-99gdc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n7sgh\" (UID: \"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:31.175846 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:31.175746    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:31.324776 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:31.324732    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"]
Jul 16 22:26:31.324970 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:26:31.324946    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25fde28_2e3d_4cf3_8a43_86f2c09a05dd.slice/crio-e9a4558e2b9c5b8f78e6c4f8411291c03e9d9aae56941b8ceb8c7d8bf28c96f3 WatchSource:0}: Error finding container e9a4558e2b9c5b8f78e6c4f8411291c03e9d9aae56941b8ceb8c7d8bf28c96f3: Status 404 returned error can't find the container with id e9a4558e2b9c5b8f78e6c4f8411291c03e9d9aae56941b8ceb8c7d8bf28c96f3
Jul 16 22:26:31.685177 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:31.685141    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh" event={"ID":"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd","Type":"ContainerStarted","Data":"eca7089c4d1fc81f96c10dea552f92041ddffaf822150a219c258a6b7ca9a13f"}
Jul 16 22:26:31.685335 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:31.685195    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh" event={"ID":"c25fde28-2e3d-4cf3-8a43-86f2c09a05dd","Type":"ContainerStarted","Data":"e9a4558e2b9c5b8f78e6c4f8411291c03e9d9aae56941b8ceb8c7d8bf28c96f3"}
Jul 16 22:26:31.720951 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:31.720894    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh" podStartSLOduration=1.559892031 podStartE2EDuration="1.720878618s" totalImagesPullingTime="160.986587ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:26:31.713734978 +0000 UTC m=+584.386993985" watchObservedRunningTime="2026-07-16 22:26:31.720878618 +0000 UTC m=+584.394137625"
Jul 16 22:26:31.900257 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:31.900219    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:35.432574 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:35.432537    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:35.433030 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:35.432609    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:35.445447 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:35.445419    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:35.715325 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:35.715248    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tdq9s"
Jul 16 22:26:38.280719 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.280688    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"]
Jul 16 22:26:38.283264 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.283247    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.285411 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.285391    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 16 22:26:38.293417 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.293391    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"]
Jul 16 22:26:38.309162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.309133    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:38.346369 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.346334    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.346369 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.346367    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3facf248-f2fe-486e-b35d-96cedba03e59-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.346608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.346393    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpbfj\" (UniqueName: \"kubernetes.io/projected/3facf248-f2fe-486e-b35d-96cedba03e59-kube-api-access-tpbfj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.346608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.346526    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.346608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.346557    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.346608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.346575    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.447625 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.447592    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.447764 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.447661    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.447764 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.447689    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.447764 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.447730    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.447764 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.447755    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3facf248-f2fe-486e-b35d-96cedba03e59-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.447959 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.447792    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpbfj\" (UniqueName: \"kubernetes.io/projected/3facf248-f2fe-486e-b35d-96cedba03e59-kube-api-access-tpbfj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.448050 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.448031    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.448140 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.448085    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.448201 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.448150    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.450213 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.450192    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3facf248-f2fe-486e-b35d-96cedba03e59-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.450478 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.450459    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3facf248-f2fe-486e-b35d-96cedba03e59-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.456064 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.456044    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpbfj\" (UniqueName: \"kubernetes.io/projected/3facf248-f2fe-486e-b35d-96cedba03e59-kube-api-access-tpbfj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt\" (UID: \"3facf248-f2fe-486e-b35d-96cedba03e59\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.594933 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.594896    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:38.733497 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:38.733456    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"]
Jul 16 22:26:38.757812 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:26:38.757779    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3facf248_f2fe_486e_b35d_96cedba03e59.slice/crio-a81b067ba034d43bfce0d9bf574a013fa74adb01038c1a2e2898adc1ab19dc0c WatchSource:0}: Error finding container a81b067ba034d43bfce0d9bf574a013fa74adb01038c1a2e2898adc1ab19dc0c: Status 404 returned error can't find the container with id a81b067ba034d43bfce0d9bf574a013fa74adb01038c1a2e2898adc1ab19dc0c
Jul 16 22:26:39.733879 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:39.733848    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt" event={"ID":"3facf248-f2fe-486e-b35d-96cedba03e59","Type":"ContainerStarted","Data":"e2b5f30d0321f9de9bbb58f71cf5ec78f428409df2f3c3113744117c47286524"}
Jul 16 22:26:39.734272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:39.733902    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt" event={"ID":"3facf248-f2fe-486e-b35d-96cedba03e59","Type":"ContainerStarted","Data":"a81b067ba034d43bfce0d9bf574a013fa74adb01038c1a2e2898adc1ab19dc0c"}
Jul 16 22:26:39.754057 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:39.754007    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt" podStartSLOduration=1.594350889 podStartE2EDuration="1.753991316s" totalImagesPullingTime="159.640427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:26:39.752727242 +0000 UTC m=+592.425986261" watchObservedRunningTime="2026-07-16 22:26:39.753991316 +0000 UTC m=+592.427250321"
Jul 16 22:26:40.489134 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:40.489081    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:41.176548 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:41.176492    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:41.176938 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:41.176576    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:41.188477 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:41.188451    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:41.754149 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:41.754106    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n7sgh"
Jul 16 22:26:47.751297 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:47.751264    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:26:47.752010 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:47.751994    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:26:47.755705 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:47.755681    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:26:47.756005 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:47.755991    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:26:48.595525 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:48.595495    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:48.597540 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:48.595558    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:48.608074 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:48.608051    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:26:48.786709 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:26:48.786684    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt"
Jul 16 22:27:19.661631 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:27:19.661533    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:51.950993 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:27:51.950953    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:52.048566 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:27:52.048530    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:32.628044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:32.628007    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:32.684468 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:32.684427    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:43.363924 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:43.363837    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:43.465995 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:43.465963    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:44.448439 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:44.448390    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:44.554319 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:44.554267    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:47.560816 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:47.560777    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:47.662851 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:28:47.662820    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:17.666241 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:17.666203    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:17.767315 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:17.767282    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:28.899423 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:28.899379    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-sjhb7"]
Jul 16 22:29:28.899998 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:28.899639    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79cbb885f5-sjhb7" podUID="553bb3ce-7944-4fd1-ada2-5c26cb435037" containerName="manager" containerID="cri-o://5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe" gracePeriod=30
Jul 16 22:29:29.158237 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.158170    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:29:29.280899 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.280863    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hknk\" (UniqueName: \"kubernetes.io/projected/553bb3ce-7944-4fd1-ada2-5c26cb435037-kube-api-access-8hknk\") pod \"553bb3ce-7944-4fd1-ada2-5c26cb435037\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") "
Jul 16 22:29:29.281076 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.280993    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/553bb3ce-7944-4fd1-ada2-5c26cb435037-webhook-cert\") pod \"553bb3ce-7944-4fd1-ada2-5c26cb435037\" (UID: \"553bb3ce-7944-4fd1-ada2-5c26cb435037\") "
Jul 16 22:29:29.283195 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.283165    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/553bb3ce-7944-4fd1-ada2-5c26cb435037-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "553bb3ce-7944-4fd1-ada2-5c26cb435037" (UID: "553bb3ce-7944-4fd1-ada2-5c26cb435037"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:29:29.283312 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.283240    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/553bb3ce-7944-4fd1-ada2-5c26cb435037-kube-api-access-8hknk" (OuterVolumeSpecName: "kube-api-access-8hknk") pod "553bb3ce-7944-4fd1-ada2-5c26cb435037" (UID: "553bb3ce-7944-4fd1-ada2-5c26cb435037"). InnerVolumeSpecName "kube-api-access-8hknk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:29:29.382211 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.382168    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hknk\" (UniqueName: \"kubernetes.io/projected/553bb3ce-7944-4fd1-ada2-5c26cb435037-kube-api-access-8hknk\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:29:29.382211 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.382201    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/553bb3ce-7944-4fd1-ada2-5c26cb435037-webhook-cert\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:29:29.578924 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.578892    2556 generic.go:356] "Generic (PLEG): container finished" podID="553bb3ce-7944-4fd1-ada2-5c26cb435037" containerID="5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe" exitCode=0
Jul 16 22:29:29.579136 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.578939    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-sjhb7"
Jul 16 22:29:29.579136 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.578971    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-sjhb7" event={"ID":"553bb3ce-7944-4fd1-ada2-5c26cb435037","Type":"ContainerDied","Data":"5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe"}
Jul 16 22:29:29.579136 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.579042    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-sjhb7" event={"ID":"553bb3ce-7944-4fd1-ada2-5c26cb435037","Type":"ContainerDied","Data":"04d7ba8b8ef0f4564dc584589b92bcbfb98667bbf42e75f01814159b74167329"}
Jul 16 22:29:29.579136 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.579081    2556 scope.go:117] "RemoveContainer" containerID="5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe"
Jul 16 22:29:29.588634 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.588617    2556 scope.go:117] "RemoveContainer" containerID="5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe"
Jul 16 22:29:29.588936 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:29:29.588915    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe\": container with ID starting with 5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe not found: ID does not exist" containerID="5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe"
Jul 16 22:29:29.589003 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.588945    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe"} err="failed to get container status \"5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe\": rpc error: code = NotFound desc = could not find container \"5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe\": container with ID starting with 5a7c61f6d155972bb4ab3e60be524628ef6f43b964c11ea66eb9ceda0c0915fe not found: ID does not exist"
Jul 16 22:29:29.605765 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.604626    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-sjhb7"]
Jul 16 22:29:29.611573 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.611546    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-sjhb7"]
Jul 16 22:29:29.721050 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.721011    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:29.786897 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.786843    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:29.812636 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:29:29.812602    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="553bb3ce-7944-4fd1-ada2-5c26cb435037" path="/var/lib/kubelet/pods/553bb3ce-7944-4fd1-ada2-5c26cb435037/volumes"
Jul 16 22:30:30.411892 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.411856    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79cbb885f5-9mtnp"]
Jul 16 22:30:30.412346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.412288    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553bb3ce-7944-4fd1-ada2-5c26cb435037" containerName="manager"
Jul 16 22:30:30.412346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.412300    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="553bb3ce-7944-4fd1-ada2-5c26cb435037" containerName="manager"
Jul 16 22:30:30.412438 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.412392    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="553bb3ce-7944-4fd1-ada2-5c26cb435037" containerName="manager"
Jul 16 22:30:30.415523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.415507    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.418931 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.418897    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mk6jb\""
Jul 16 22:30:30.419462 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.419443    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 22:30:30.419542 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.419446    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 22:30:30.429376 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.429355    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-9mtnp"]
Jul 16 22:30:30.436594 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.436573    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-webhook-cert\") pod \"maas-controller-79cbb885f5-9mtnp\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") " pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.436687 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.436630    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8br5\" (UniqueName: \"kubernetes.io/projected/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-kube-api-access-v8br5\") pod \"maas-controller-79cbb885f5-9mtnp\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") " pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.538057 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.538020    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8br5\" (UniqueName: \"kubernetes.io/projected/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-kube-api-access-v8br5\") pod \"maas-controller-79cbb885f5-9mtnp\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") " pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.538245 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.538105    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-webhook-cert\") pod \"maas-controller-79cbb885f5-9mtnp\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") " pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.540690 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.540671    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-webhook-cert\") pod \"maas-controller-79cbb885f5-9mtnp\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") " pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.548075 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.548051    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8br5\" (UniqueName: \"kubernetes.io/projected/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-kube-api-access-v8br5\") pod \"maas-controller-79cbb885f5-9mtnp\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") " pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.725835 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.725736    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:30.867352 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.867329    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-9mtnp"]
Jul 16 22:30:30.868974 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:30:30.868938    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e8ccd5_6062_4a94_ba50_76f638bc8bbe.slice/crio-890d9f19f7f23922d39df374453f5a147fcb939584df8817a7a93718bfb1b293 WatchSource:0}: Error finding container 890d9f19f7f23922d39df374453f5a147fcb939584df8817a7a93718bfb1b293: Status 404 returned error can't find the container with id 890d9f19f7f23922d39df374453f5a147fcb939584df8817a7a93718bfb1b293
Jul 16 22:30:30.870207 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.870191    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:30:30.883961 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:30.883932    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-9mtnp" event={"ID":"68e8ccd5-6062-4a94-ba50-76f638bc8bbe","Type":"ContainerStarted","Data":"890d9f19f7f23922d39df374453f5a147fcb939584df8817a7a93718bfb1b293"}
Jul 16 22:30:31.897456 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:31.897412    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-9mtnp" event={"ID":"68e8ccd5-6062-4a94-ba50-76f638bc8bbe","Type":"ContainerStarted","Data":"dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f"}
Jul 16 22:30:31.897921 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:31.897517    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:31.917914 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:31.917867    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79cbb885f5-9mtnp" podStartSLOduration=1.271014017 podStartE2EDuration="1.917853391s" totalImagesPullingTime="646.839374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:30:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:30:31.915891348 +0000 UTC m=+824.589150359" watchObservedRunningTime="2026-07-16 22:30:31.917853391 +0000 UTC m=+824.591112399"
Jul 16 22:30:42.905433 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:42.905401    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:47.442008 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.441969    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:47.535592 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.535551    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-9mtnp"]
Jul 16 22:30:47.535837 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.535794    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79cbb885f5-9mtnp" podUID="68e8ccd5-6062-4a94-ba50-76f638bc8bbe" containerName="manager" containerID="cri-o://dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f" gracePeriod=30
Jul 16 22:30:47.791240 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.791216    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:47.900047 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.900008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-webhook-cert\") pod \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") "
Jul 16 22:30:47.900245 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.900101    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8br5\" (UniqueName: \"kubernetes.io/projected/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-kube-api-access-v8br5\") pod \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\" (UID: \"68e8ccd5-6062-4a94-ba50-76f638bc8bbe\") "
Jul 16 22:30:47.902260 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.902233    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "68e8ccd5-6062-4a94-ba50-76f638bc8bbe" (UID: "68e8ccd5-6062-4a94-ba50-76f638bc8bbe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:30:47.902353 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.902273    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-kube-api-access-v8br5" (OuterVolumeSpecName: "kube-api-access-v8br5") pod "68e8ccd5-6062-4a94-ba50-76f638bc8bbe" (UID: "68e8ccd5-6062-4a94-ba50-76f638bc8bbe"). InnerVolumeSpecName "kube-api-access-v8br5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:30:47.985870 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.985767    2556 generic.go:356] "Generic (PLEG): container finished" podID="68e8ccd5-6062-4a94-ba50-76f638bc8bbe" containerID="dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f" exitCode=0
Jul 16 22:30:47.985870 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.985819    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-9mtnp"
Jul 16 22:30:47.986223 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.985813    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-9mtnp" event={"ID":"68e8ccd5-6062-4a94-ba50-76f638bc8bbe","Type":"ContainerDied","Data":"dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f"}
Jul 16 22:30:47.986223 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.985923    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-9mtnp" event={"ID":"68e8ccd5-6062-4a94-ba50-76f638bc8bbe","Type":"ContainerDied","Data":"890d9f19f7f23922d39df374453f5a147fcb939584df8817a7a93718bfb1b293"}
Jul 16 22:30:47.986223 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.985958    2556 scope.go:117] "RemoveContainer" containerID="dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f"
Jul 16 22:30:47.995084 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.995067    2556 scope.go:117] "RemoveContainer" containerID="dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f"
Jul 16 22:30:47.995347 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:30:47.995325    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f\": container with ID starting with dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f not found: ID does not exist" containerID="dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f"
Jul 16 22:30:47.995402 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:47.995356    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f"} err="failed to get container status \"dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f\": rpc error: code = NotFound desc = could not find container \"dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f\": container with ID starting with dd20d6dbf03d31094431650c558259f4ede4f355e2363ff1dc6288b0c2d4937f not found: ID does not exist"
Jul 16 22:30:48.000976 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.000959    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8br5\" (UniqueName: \"kubernetes.io/projected/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-kube-api-access-v8br5\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:30:48.001038 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.000978    2556 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68e8ccd5-6062-4a94-ba50-76f638bc8bbe-webhook-cert\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:30:48.008405 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.008385    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-9mtnp"]
Jul 16 22:30:48.013965 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.013946    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-9mtnp"]
Jul 16 22:30:48.680844 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.680808    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:48.788069 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.788023    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79cbb885f5-zp56f"]
Jul 16 22:30:48.789056 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.789031    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68e8ccd5-6062-4a94-ba50-76f638bc8bbe" containerName="manager"
Jul 16 22:30:48.789056 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.789059    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e8ccd5-6062-4a94-ba50-76f638bc8bbe" containerName="manager"
Jul 16 22:30:48.789275 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.789261    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="68e8ccd5-6062-4a94-ba50-76f638bc8bbe" containerName="manager"
Jul 16 22:30:48.797252 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.797228    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:48.800583 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.800548    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-zp56f"]
Jul 16 22:30:48.800772 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.800748    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mk6jb\""
Jul 16 22:30:48.800964 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.800716    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 22:30:48.801350 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.801329    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 22:30:48.909381 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.909341    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njt8d\" (UniqueName: \"kubernetes.io/projected/c4283ce5-4ba4-4ce6-b531-31ba1610d50e-kube-api-access-njt8d\") pod \"maas-controller-79cbb885f5-zp56f\" (UID: \"c4283ce5-4ba4-4ce6-b531-31ba1610d50e\") " pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:48.909607 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:48.909534    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4283ce5-4ba4-4ce6-b531-31ba1610d50e-webhook-cert\") pod \"maas-controller-79cbb885f5-zp56f\" (UID: \"c4283ce5-4ba4-4ce6-b531-31ba1610d50e\") " pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:49.010493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.010391    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njt8d\" (UniqueName: \"kubernetes.io/projected/c4283ce5-4ba4-4ce6-b531-31ba1610d50e-kube-api-access-njt8d\") pod \"maas-controller-79cbb885f5-zp56f\" (UID: \"c4283ce5-4ba4-4ce6-b531-31ba1610d50e\") " pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:49.010493 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.010450    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4283ce5-4ba4-4ce6-b531-31ba1610d50e-webhook-cert\") pod \"maas-controller-79cbb885f5-zp56f\" (UID: \"c4283ce5-4ba4-4ce6-b531-31ba1610d50e\") " pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:49.012993 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.012963    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4283ce5-4ba4-4ce6-b531-31ba1610d50e-webhook-cert\") pod \"maas-controller-79cbb885f5-zp56f\" (UID: \"c4283ce5-4ba4-4ce6-b531-31ba1610d50e\") " pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:49.019688 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.019664    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njt8d\" (UniqueName: \"kubernetes.io/projected/c4283ce5-4ba4-4ce6-b531-31ba1610d50e-kube-api-access-njt8d\") pod \"maas-controller-79cbb885f5-zp56f\" (UID: \"c4283ce5-4ba4-4ce6-b531-31ba1610d50e\") " pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:49.111390 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.111346    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:49.249404 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.249376    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79cbb885f5-zp56f"]
Jul 16 22:30:49.251558 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:30:49.251528    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4283ce5_4ba4_4ce6_b531_31ba1610d50e.slice/crio-1ea95f346f2762c1c69aec57f890685ae9112243fabf1474e3c1e2d86761ccd9 WatchSource:0}: Error finding container 1ea95f346f2762c1c69aec57f890685ae9112243fabf1474e3c1e2d86761ccd9: Status 404 returned error can't find the container with id 1ea95f346f2762c1c69aec57f890685ae9112243fabf1474e3c1e2d86761ccd9
Jul 16 22:30:49.812097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.812062    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e8ccd5-6062-4a94-ba50-76f638bc8bbe" path="/var/lib/kubelet/pods/68e8ccd5-6062-4a94-ba50-76f638bc8bbe/volumes"
Jul 16 22:30:49.998397 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.998298    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-zp56f" event={"ID":"c4283ce5-4ba4-4ce6-b531-31ba1610d50e","Type":"ContainerStarted","Data":"8c6ec6622f567869b7b86dc134eeaabf66afdb6b0badd93a5a4f980e9cd8bd72"}
Jul 16 22:30:49.998397 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.998350    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79cbb885f5-zp56f" event={"ID":"c4283ce5-4ba4-4ce6-b531-31ba1610d50e","Type":"ContainerStarted","Data":"1ea95f346f2762c1c69aec57f890685ae9112243fabf1474e3c1e2d86761ccd9"}
Jul 16 22:30:49.998625 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:49.998425    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:30:50.015722 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:30:50.015664    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79cbb885f5-zp56f" podStartSLOduration=1.638323357 podStartE2EDuration="2.015641037s" totalImagesPullingTime="377.31768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:30:50.013730259 +0000 UTC m=+842.686989267" watchObservedRunningTime="2026-07-16 22:30:50.015641037 +0000 UTC m=+842.688900046"
Jul 16 22:31:01.007172 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:01.007141    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79cbb885f5-zp56f"
Jul 16 22:31:27.281528 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:27.281487    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:27.385974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:27.385900    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:27.747871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:27.747833    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:30.187370 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:30.187333    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:33.314974 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:33.314935    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:33.449160 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:33.449089    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:33.712644 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:33.712548    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:33.812235 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:33.812197    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:39.942435 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:39.942351    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:40.042752 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:40.042709    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:47.784036 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:47.784001    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:31:47.785630 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:47.785608    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:31:47.788497 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:47.788475    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:31:47.789679 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:31:47.789664    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:32:27.817998 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:32:27.817965    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:36.848346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:33:36.848261    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:05.129772 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:05.129732    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:41.139624 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:41.139587    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"]
Jul 16 22:35:41.140239 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:41.139790    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49" podUID="0e3c7900-a966-4e00-a239-7c862445450b" containerName="manager" containerID="cri-o://473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c" gracePeriod=10
Jul 16 22:35:44.089958 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.089933    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:35:44.092002 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.091982    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7c6z\" (UniqueName: \"kubernetes.io/projected/0e3c7900-a966-4e00-a239-7c862445450b-kube-api-access-z7c6z\") pod \"0e3c7900-a966-4e00-a239-7c862445450b\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") "
Jul 16 22:35:44.092057 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.092042    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e3c7900-a966-4e00-a239-7c862445450b-extensions-socket-volume\") pod \"0e3c7900-a966-4e00-a239-7c862445450b\" (UID: \"0e3c7900-a966-4e00-a239-7c862445450b\") "
Jul 16 22:35:44.092408 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.092387    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e3c7900-a966-4e00-a239-7c862445450b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0e3c7900-a966-4e00-a239-7c862445450b" (UID: "0e3c7900-a966-4e00-a239-7c862445450b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:35:44.094195 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.094172    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e3c7900-a966-4e00-a239-7c862445450b-kube-api-access-z7c6z" (OuterVolumeSpecName: "kube-api-access-z7c6z") pod "0e3c7900-a966-4e00-a239-7c862445450b" (UID: "0e3c7900-a966-4e00-a239-7c862445450b"). InnerVolumeSpecName "kube-api-access-z7c6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:35:44.193410 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.193360    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7c6z\" (UniqueName: \"kubernetes.io/projected/0e3c7900-a966-4e00-a239-7c862445450b-kube-api-access-z7c6z\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:35:44.193410 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.193400    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e3c7900-a966-4e00-a239-7c862445450b-extensions-socket-volume\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:35:44.492294 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.492261    2556 generic.go:356] "Generic (PLEG): container finished" podID="0e3c7900-a966-4e00-a239-7c862445450b" containerID="473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c" exitCode=0
Jul 16 22:35:44.492489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.492305    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"
Jul 16 22:35:44.492489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.492357    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49" event={"ID":"0e3c7900-a966-4e00-a239-7c862445450b","Type":"ContainerDied","Data":"473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c"}
Jul 16 22:35:44.492489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.492405    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49" event={"ID":"0e3c7900-a966-4e00-a239-7c862445450b","Type":"ContainerDied","Data":"49ce5084b59770b6f3add5a5b96d7836207141f5766752966e82d0cd868d6ee4"}
Jul 16 22:35:44.492489 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.492427    2556 scope.go:117] "RemoveContainer" containerID="473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c"
Jul 16 22:35:44.502931 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.502915    2556 scope.go:117] "RemoveContainer" containerID="473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c"
Jul 16 22:35:44.503224 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:35:44.503205    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c\": container with ID starting with 473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c not found: ID does not exist" containerID="473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c"
Jul 16 22:35:44.503295 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.503231    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c"} err="failed to get container status \"473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c\": rpc error: code = NotFound desc = could not find container \"473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c\": container with ID starting with 473ecfe1522287011a3a93c65e8902641cc7bc2a73e83e164eb805b40ddec63c not found: ID does not exist"
Jul 16 22:35:44.518029 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.518006    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"]
Jul 16 22:35:44.522080 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:44.522059    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mdg49"]
Jul 16 22:35:45.812257 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:45.812222    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e3c7900-a966-4e00-a239-7c862445450b" path="/var/lib/kubelet/pods/0e3c7900-a966-4e00-a239-7c862445450b/volumes"
Jul 16 22:35:55.544568 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:55.544528    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:55.618173 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:35:55.618134    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:42.293924 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.293895    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"]
Jul 16 22:36:42.294373 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.294298    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e3c7900-a966-4e00-a239-7c862445450b" containerName="manager"
Jul 16 22:36:42.294373 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.294309    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e3c7900-a966-4e00-a239-7c862445450b" containerName="manager"
Jul 16 22:36:42.294373 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.294369    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e3c7900-a966-4e00-a239-7c862445450b" containerName="manager"
Jul 16 22:36:42.297311 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.297292    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.305921 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.305902    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-shwv8\""
Jul 16 22:36:42.315796 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.315767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rf6j\" (UniqueName: \"kubernetes.io/projected/24ac699f-66cc-4bc2-a22f-1087168aa9e2-kube-api-access-5rf6j\") pod \"kuadrant-operator-controller-manager-86878c769f-cdnbt\" (UID: \"24ac699f-66cc-4bc2-a22f-1087168aa9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.315882 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.315845    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ac699f-66cc-4bc2-a22f-1087168aa9e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cdnbt\" (UID: \"24ac699f-66cc-4bc2-a22f-1087168aa9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.319361 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.319339    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"]
Jul 16 22:36:42.417281 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.417244    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rf6j\" (UniqueName: \"kubernetes.io/projected/24ac699f-66cc-4bc2-a22f-1087168aa9e2-kube-api-access-5rf6j\") pod \"kuadrant-operator-controller-manager-86878c769f-cdnbt\" (UID: \"24ac699f-66cc-4bc2-a22f-1087168aa9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.417536 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.417354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ac699f-66cc-4bc2-a22f-1087168aa9e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cdnbt\" (UID: \"24ac699f-66cc-4bc2-a22f-1087168aa9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.417745 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.417726    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24ac699f-66cc-4bc2-a22f-1087168aa9e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cdnbt\" (UID: \"24ac699f-66cc-4bc2-a22f-1087168aa9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.427369 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.427349    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rf6j\" (UniqueName: \"kubernetes.io/projected/24ac699f-66cc-4bc2-a22f-1087168aa9e2-kube-api-access-5rf6j\") pod \"kuadrant-operator-controller-manager-86878c769f-cdnbt\" (UID: \"24ac699f-66cc-4bc2-a22f-1087168aa9e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.607190 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.607074    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:42.757742 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.757716    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"]
Jul 16 22:36:42.760131 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:36:42.760086    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ac699f_66cc_4bc2_a22f_1087168aa9e2.slice/crio-2fedbca542c056d678e00e14761670c89ef90e4a843a0820103932559a92d997 WatchSource:0}: Error finding container 2fedbca542c056d678e00e14761670c89ef90e4a843a0820103932559a92d997: Status 404 returned error can't find the container with id 2fedbca542c056d678e00e14761670c89ef90e4a843a0820103932559a92d997
Jul 16 22:36:42.781870 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:42.781844    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt" event={"ID":"24ac699f-66cc-4bc2-a22f-1087168aa9e2","Type":"ContainerStarted","Data":"2fedbca542c056d678e00e14761670c89ef90e4a843a0820103932559a92d997"}
Jul 16 22:36:43.788443 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:43.788395    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt" event={"ID":"24ac699f-66cc-4bc2-a22f-1087168aa9e2","Type":"ContainerStarted","Data":"b0d9eb62733fb125bbd8253cc447daf78022b59d27c554aa0dab03b536f965ab"}
Jul 16 22:36:43.788835 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:43.788470    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:36:43.815715 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:43.815663    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt" podStartSLOduration=1.815646685 podStartE2EDuration="1.815646685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:36:43.812844161 +0000 UTC m=+1196.486103169" watchObservedRunningTime="2026-07-16 22:36:43.815646685 +0000 UTC m=+1196.488905694"
Jul 16 22:36:47.816772 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:47.816744    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:36:47.818729 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:47.818707    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:36:47.820853 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:47.820836    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:36:47.825908 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:47.825889    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:36:54.795585 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:36:54.795553    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cdnbt"
Jul 16 22:38:27.850064 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:38:27.850028    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:29.172740 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:38:29.172703    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:41:47.851076 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:41:47.851043    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:41:47.854548 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:41:47.854525    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:41:47.855608 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:41:47.855587    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:41:47.858809 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:41:47.858790    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:42:34.889535 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:42:34.889458    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:41.702467 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:42:41.702421    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:49.958726 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:42:49.958682    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:55.078606 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:42:55.078570    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:58.025244 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:42:58.025206    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:58.150842 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:42:58.150810    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:58.469161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:42:58.469086    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:04.851202 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:04.851168    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:04.982189 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:04.982153    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:05.167971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:05.167882    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:05.289797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:05.289754    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:06.420401 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.420364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:06.582542 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.582512    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:06.672290 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.672202    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"]
Jul 16 22:43:06.676393 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.676373    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.679152 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.679125    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5cb4a56a-serving-cert\""
Jul 16 22:43:06.695652 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.695620    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 22:43:06.762204 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.762171    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"]
Jul 16 22:43:06.801226 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.801186    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466122f1-7446-46a2-adec-b1988cbe5974-openshift-service-ca\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.801419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.801270    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj887\" (UniqueName: \"kubernetes.io/projected/466122f1-7446-46a2-adec-b1988cbe5974-kube-api-access-jj887\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.801419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.801295    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.902568 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.902522    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj887\" (UniqueName: \"kubernetes.io/projected/466122f1-7446-46a2-adec-b1988cbe5974-kube-api-access-jj887\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.902568 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.902572    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.902850 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.902651    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466122f1-7446-46a2-adec-b1988cbe5974-openshift-service-ca\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.902850 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:06.902748    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-serving-cert: secret "maas-api-e2e-ait-5cb4a56a-serving-cert" not found
Jul 16 22:43:06.902850 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:06.902826    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls podName:466122f1-7446-46a2-adec-b1988cbe5974 nodeName:}" failed. No retries permitted until 2026-07-16 22:43:07.402808661 +0000 UTC m=+1580.076067647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls") pod "maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" (UID: "466122f1-7446-46a2-adec-b1988cbe5974") : secret "maas-api-e2e-ait-5cb4a56a-serving-cert" not found
Jul 16 22:43:06.903288 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.903270    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466122f1-7446-46a2-adec-b1988cbe5974-openshift-service-ca\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.931167 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.931080    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj887\" (UniqueName: \"kubernetes.io/projected/466122f1-7446-46a2-adec-b1988cbe5974-kube-api-access-jj887\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:06.945186 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:06.945143    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:07.093103 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.093064    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:07.407045 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.407008    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:07.409826 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.409797    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls\") pod \"maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:07.586613 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.586576    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:07.644805 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.644767    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:07.770747 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.770706    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:07.959949 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.959925    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"]
Jul 16 22:43:07.961967 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:43:07.961939    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod466122f1_7446_46a2_adec_b1988cbe5974.slice/crio-834ab186b695a0aa64ef434ec79191f305c6f51ba60692e6aa59590aed3a7b85 WatchSource:0}: Error finding container 834ab186b695a0aa64ef434ec79191f305c6f51ba60692e6aa59590aed3a7b85: Status 404 returned error can't find the container with id 834ab186b695a0aa64ef434ec79191f305c6f51ba60692e6aa59590aed3a7b85
Jul 16 22:43:07.963339 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:07.963308    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:43:08.708934 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:08.708902    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" event={"ID":"466122f1-7446-46a2-adec-b1988cbe5974","Type":"ContainerStarted","Data":"834ab186b695a0aa64ef434ec79191f305c6f51ba60692e6aa59590aed3a7b85"}
Jul 16 22:43:09.072337 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:09.072300    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:09.715888 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:09.715852    2556 generic.go:356] "Generic (PLEG): container finished" podID="466122f1-7446-46a2-adec-b1988cbe5974" containerID="443f3cfcbfc630d3b512d2618ba4c3dfa42531dcf530f1fd9208579250050377" exitCode=1
Jul 16 22:43:09.716341 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:09.715941    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" event={"ID":"466122f1-7446-46a2-adec-b1988cbe5974","Type":"ContainerDied","Data":"443f3cfcbfc630d3b512d2618ba4c3dfa42531dcf530f1fd9208579250050377"}
Jul 16 22:43:09.716341 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:09.716177    2556 scope.go:117] "RemoveContainer" containerID="443f3cfcbfc630d3b512d2618ba4c3dfa42531dcf530f1fd9208579250050377"
Jul 16 22:43:10.724584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:10.724488    2556 generic.go:356] "Generic (PLEG): container finished" podID="466122f1-7446-46a2-adec-b1988cbe5974" containerID="4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9" exitCode=1
Jul 16 22:43:10.724584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:10.724547    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" event={"ID":"466122f1-7446-46a2-adec-b1988cbe5974","Type":"ContainerDied","Data":"4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9"}
Jul 16 22:43:10.725100 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:10.724597    2556 scope.go:117] "RemoveContainer" containerID="443f3cfcbfc630d3b512d2618ba4c3dfa42531dcf530f1fd9208579250050377"
Jul 16 22:43:10.725100 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:10.724837    2556 scope.go:117] "RemoveContainer" containerID="4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9"
Jul 16 22:43:10.725100 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:10.725044    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:43:11.731358 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:11.731333    2556 scope.go:117] "RemoveContainer" containerID="4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9"
Jul 16 22:43:11.731766 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:11.731521    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:43:12.586724 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:12.586689    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:12.736549 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:12.736513    2556 scope.go:117] "RemoveContainer" containerID="4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9"
Jul 16 22:43:12.737011 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:12.736765    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:43:17.587626 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:17.587586    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:17.588041 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:17.587981    2556 scope.go:117] "RemoveContainer" containerID="4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9"
Jul 16 22:43:17.588218 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:17.588197    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:43:26.582830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:26.582780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:26.656381 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:26.656342    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:31.805939 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:31.805900    2556 scope.go:117] "RemoveContainer" containerID="4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9"
Jul 16 22:43:32.834575 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:32.834482    2556 generic.go:356] "Generic (PLEG): container finished" podID="466122f1-7446-46a2-adec-b1988cbe5974" containerID="96ffa716e6d3aad499285c45f301374847a03ea1ac6b59866506319d2b23dd5d" exitCode=1
Jul 16 22:43:32.834575 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:32.834531    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" event={"ID":"466122f1-7446-46a2-adec-b1988cbe5974","Type":"ContainerDied","Data":"96ffa716e6d3aad499285c45f301374847a03ea1ac6b59866506319d2b23dd5d"}
Jul 16 22:43:32.835061 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:32.834594    2556 scope.go:117] "RemoveContainer" containerID="4c8ddcffa0c68fcde47592816715fd7f1d4d0592b0647884ae158eaa7ddfdac9"
Jul 16 22:43:32.835061 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:32.834936    2556 scope.go:117] "RemoveContainer" containerID="96ffa716e6d3aad499285c45f301374847a03ea1ac6b59866506319d2b23dd5d"
Jul 16 22:43:32.835236 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:32.835212    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:43:37.587213 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:37.587162    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:37.587737 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:37.587236    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:43:37.587737 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:37.587582    2556 scope.go:117] "RemoveContainer" containerID="96ffa716e6d3aad499285c45f301374847a03ea1ac6b59866506319d2b23dd5d"
Jul 16 22:43:37.587832 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:37.587759    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:43:50.805602 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:43:50.805517    2556 scope.go:117] "RemoveContainer" containerID="96ffa716e6d3aad499285c45f301374847a03ea1ac6b59866506319d2b23dd5d"
Jul 16 22:43:50.805971 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:43:50.805699    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:02.806791 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:02.806752    2556 scope.go:117] "RemoveContainer" containerID="96ffa716e6d3aad499285c45f301374847a03ea1ac6b59866506319d2b23dd5d"
Jul 16 22:44:03.996087 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:03.996052    2556 generic.go:356] "Generic (PLEG): container finished" podID="466122f1-7446-46a2-adec-b1988cbe5974" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2" exitCode=1
Jul 16 22:44:03.996521 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:03.996155    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" event={"ID":"466122f1-7446-46a2-adec-b1988cbe5974","Type":"ContainerDied","Data":"3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"}
Jul 16 22:44:03.996521 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:03.996245    2556 scope.go:117] "RemoveContainer" containerID="96ffa716e6d3aad499285c45f301374847a03ea1ac6b59866506319d2b23dd5d"
Jul 16 22:44:03.996669 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:03.996656    2556 scope.go:117] "RemoveContainer" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"
Jul 16 22:44:03.996883 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:03.996864    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:07.587359 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:07.587321    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:44:07.587790 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:07.587388    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:44:07.587790 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:07.587701    2556 scope.go:117] "RemoveContainer" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"
Jul 16 22:44:07.587890 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:07.587870    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:13.289279 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:13.289242    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:20.806371 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:20.806334    2556 scope.go:117] "RemoveContainer" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"
Jul 16 22:44:20.806794 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:20.806533    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:31.805744 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:31.805712    2556 scope.go:117] "RemoveContainer" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"
Jul 16 22:44:31.806278 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:31.805897    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:42.805895 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:42.805860    2556 scope.go:117] "RemoveContainer" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"
Jul 16 22:44:42.806352 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:42.806060    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:55.806510 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:55.806477    2556 scope.go:117] "RemoveContainer" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"
Jul 16 22:44:57.285358 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:57.285323    2556 generic.go:356] "Generic (PLEG): container finished" podID="466122f1-7446-46a2-adec-b1988cbe5974" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727" exitCode=1
Jul 16 22:44:57.285787 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:57.285399    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" event={"ID":"466122f1-7446-46a2-adec-b1988cbe5974","Type":"ContainerDied","Data":"7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"}
Jul 16 22:44:57.285787 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:57.285461    2556 scope.go:117] "RemoveContainer" containerID="3f6af52eca5c3c231f023e9e994f5cee37b5f53896b823ffed7312903bd43bd2"
Jul 16 22:44:57.285896 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:57.285829    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:44:57.286069 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:57.286044    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:57.586786 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:57.586752    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:44:57.586984 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:57.586852    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:44:58.292284 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:58.292253    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:44:58.292734 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:58.292510    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:44:59.297514 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:44:59.297490    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:44:59.297904 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:44:59.297689    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:45:12.806166 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:45:12.806049    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:45:12.806730 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:45:12.806314    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:45:25.806278 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:45:25.806244    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:45:25.806809 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:45:25.806484    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:45:39.805732 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:45:39.805703    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:45:39.806319 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:45:39.805915    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:45:44.226044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:45:44.226008    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:44.314434 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:45:44.314397    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:54.806364 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:45:54.806333    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:45:54.806848 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:45:54.806555    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:46:07.808731 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:07.808694    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:46:07.809249 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:07.808936    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc_odh-ai-gateway-infra(466122f1-7446-46a2-adec-b1988cbe5974)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" podUID="466122f1-7446-46a2-adec-b1988cbe5974"
Jul 16 22:46:10.061346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:10.061311    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:10.160698 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:10.160663    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:11.433049 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:11.433007    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:11.517322 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:11.517280    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:17.216339 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.216303    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"]
Jul 16 22:46:17.257240 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.257206    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:17.361097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.361070    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:46:17.475471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.475381    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj887\" (UniqueName: \"kubernetes.io/projected/466122f1-7446-46a2-adec-b1988cbe5974-kube-api-access-jj887\") pod \"466122f1-7446-46a2-adec-b1988cbe5974\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") "
Jul 16 22:46:17.475471 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.475443    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls\") pod \"466122f1-7446-46a2-adec-b1988cbe5974\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") "
Jul 16 22:46:17.475655 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.475480    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466122f1-7446-46a2-adec-b1988cbe5974-openshift-service-ca\") pod \"466122f1-7446-46a2-adec-b1988cbe5974\" (UID: \"466122f1-7446-46a2-adec-b1988cbe5974\") "
Jul 16 22:46:17.475851 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.475814    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/466122f1-7446-46a2-adec-b1988cbe5974-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "466122f1-7446-46a2-adec-b1988cbe5974" (UID: "466122f1-7446-46a2-adec-b1988cbe5974"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:46:17.477777 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.477752    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/466122f1-7446-46a2-adec-b1988cbe5974-kube-api-access-jj887" (OuterVolumeSpecName: "kube-api-access-jj887") pod "466122f1-7446-46a2-adec-b1988cbe5974" (UID: "466122f1-7446-46a2-adec-b1988cbe5974"). InnerVolumeSpecName "kube-api-access-jj887". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:46:17.477864 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.477843    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "466122f1-7446-46a2-adec-b1988cbe5974" (UID: "466122f1-7446-46a2-adec-b1988cbe5974"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:46:17.576395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.576354    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/466122f1-7446-46a2-adec-b1988cbe5974-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:46:17.576395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.576386    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/466122f1-7446-46a2-adec-b1988cbe5974-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:46:17.576395 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.576396    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj887\" (UniqueName: \"kubernetes.io/projected/466122f1-7446-46a2-adec-b1988cbe5974-kube-api-access-jj887\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:46:17.704583 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.704554    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"
Jul 16 22:46:17.704747 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.704590    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc" event={"ID":"466122f1-7446-46a2-adec-b1988cbe5974","Type":"ContainerDied","Data":"834ab186b695a0aa64ef434ec79191f305c6f51ba60692e6aa59590aed3a7b85"}
Jul 16 22:46:17.704747 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.704653    2556 scope.go:117] "RemoveContainer" containerID="7c4606d9b4cfde72dd1b6360607568f2b69c111393370be0cce7dbd930e1d727"
Jul 16 22:46:17.721279 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.721254    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:17.726082 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.726020    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"]
Jul 16 22:46:17.730161 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.730139    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5cb4a56a-56fc5bc457-7nkfc"]
Jul 16 22:46:17.812568 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:17.812531    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="466122f1-7446-46a2-adec-b1988cbe5974" path="/var/lib/kubelet/pods/466122f1-7446-46a2-adec-b1988cbe5974/volumes"
Jul 16 22:46:22.967078 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:22.967019    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:23.309386 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:23.309337    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:23.412373 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:23.412335    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:23.573378 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:23.573277    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:23.673197 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:23.673160    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:23.777998 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:23.777955    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:23.849485 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:23.849352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:25.124299 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.124261    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:25.211166 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211131    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"]
Jul 16 22:46:25.211639 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211620    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211741 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211643    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211741 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211675    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211741 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211686    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211741 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211702    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211741 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211710    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211984 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211813    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211984 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211827    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211984 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211838    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.211984 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.211847    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:46:25.215584 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.215556    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.218014 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.217987    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-bf38b381-serving-cert\""
Jul 16 22:46:25.218156 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.218093    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 22:46:25.225479 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.225265    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"]
Jul 16 22:46:25.242214 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.242184    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.242396 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.242223    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swmhc\" (UniqueName: \"kubernetes.io/projected/8bc10682-71a0-4e2b-872c-a7dffc915bab-kube-api-access-swmhc\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.242396 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.242244    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc10682-71a0-4e2b-872c-a7dffc915bab-openshift-service-ca\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.244366 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.244337    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:25.343481 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.343440    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.343689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.343498    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swmhc\" (UniqueName: \"kubernetes.io/projected/8bc10682-71a0-4e2b-872c-a7dffc915bab-kube-api-access-swmhc\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.343689 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:25.343603    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-serving-cert: secret "maas-api-e2e-migrate-bf38b381-serving-cert" not found
Jul 16 22:46:25.343689 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:25.343675    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls podName:8bc10682-71a0-4e2b-872c-a7dffc915bab nodeName:}" failed. No retries permitted until 2026-07-16 22:46:25.843653325 +0000 UTC m=+1778.516912311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls") pod "maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" (UID: "8bc10682-71a0-4e2b-872c-a7dffc915bab") : secret "maas-api-e2e-migrate-bf38b381-serving-cert" not found
Jul 16 22:46:25.343857 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.343698    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc10682-71a0-4e2b-872c-a7dffc915bab-openshift-service-ca\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.344448 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.344422    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc10682-71a0-4e2b-872c-a7dffc915bab-openshift-service-ca\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.356620 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.356589    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swmhc\" (UniqueName: \"kubernetes.io/projected/8bc10682-71a0-4e2b-872c-a7dffc915bab-kube-api-access-swmhc\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.406387 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.406290    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:25.746808 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.746687    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:25.847362 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.847325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:25.847585 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.847466    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:25.850043 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:25.850020    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls\") pod \"maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:26.128977 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:26.128930    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:26.288018 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:26.280551    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"]
Jul 16 22:46:26.755351 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:26.755304    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" event={"ID":"8bc10682-71a0-4e2b-872c-a7dffc915bab","Type":"ContainerStarted","Data":"3f841e93ee1cb9bb700a93e8e2ea2aa578109dda18ea5b25fef209e8c49b0aad"}
Jul 16 22:46:26.755532 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:26.755515    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:26.755599 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:26.755554    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" event={"ID":"8bc10682-71a0-4e2b-872c-a7dffc915bab","Type":"ContainerStarted","Data":"0473e51ff22f03a978cd566c76efc1dbeb22286dd0c36ef0b4a85779c7dd6838"}
Jul 16 22:46:26.786966 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:26.786829    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podStartSLOduration=1.398376503 podStartE2EDuration="1.786814451s" totalImagesPullingTime="388.437948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:46:26.785560693 +0000 UTC m=+1779.458819730" watchObservedRunningTime="2026-07-16 22:46:26.786814451 +0000 UTC m=+1779.460073459"
Jul 16 22:46:27.157185 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:27.157149    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:27.762410 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:27.762361    2556 generic.go:356] "Generic (PLEG): container finished" podID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerID="3f841e93ee1cb9bb700a93e8e2ea2aa578109dda18ea5b25fef209e8c49b0aad" exitCode=1
Jul 16 22:46:27.762607 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:27.762431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" event={"ID":"8bc10682-71a0-4e2b-872c-a7dffc915bab","Type":"ContainerDied","Data":"3f841e93ee1cb9bb700a93e8e2ea2aa578109dda18ea5b25fef209e8c49b0aad"}
Jul 16 22:46:27.762700 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:27.762685    2556 scope.go:117] "RemoveContainer" containerID="3f841e93ee1cb9bb700a93e8e2ea2aa578109dda18ea5b25fef209e8c49b0aad"
Jul 16 22:46:28.769739 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:28.769697    2556 generic.go:356] "Generic (PLEG): container finished" podID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerID="15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0" exitCode=1
Jul 16 22:46:28.770172 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:28.769781    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" event={"ID":"8bc10682-71a0-4e2b-872c-a7dffc915bab","Type":"ContainerDied","Data":"15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0"}
Jul 16 22:46:28.770172 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:28.769852    2556 scope.go:117] "RemoveContainer" containerID="3f841e93ee1cb9bb700a93e8e2ea2aa578109dda18ea5b25fef209e8c49b0aad"
Jul 16 22:46:28.770172 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:28.770135    2556 scope.go:117] "RemoveContainer" containerID="15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0"
Jul 16 22:46:28.770396 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:28.770374    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:46:29.776985 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:29.776950    2556 scope.go:117] "RemoveContainer" containerID="15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0"
Jul 16 22:46:29.777415 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:29.777176    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:46:31.756215 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:31.756183    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:31.756653 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:31.756638    2556 scope.go:117] "RemoveContainer" containerID="15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0"
Jul 16 22:46:31.756835 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:31.756815    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:46:36.129830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:36.129795    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:36.130291 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:36.130211    2556 scope.go:117] "RemoveContainer" containerID="15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0"
Jul 16 22:46:36.130398 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:36.130379    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:46:47.881798 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:47.881765    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:46:47.885866 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:47.885846    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:46:47.886267 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:47.886247    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:46:47.890195 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:47.890178    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:46:50.806216 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:50.806176    2556 scope.go:117] "RemoveContainer" containerID="15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0"
Jul 16 22:46:51.888487 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:51.888445    2556 generic.go:356] "Generic (PLEG): container finished" podID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerID="d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6" exitCode=1
Jul 16 22:46:51.888868 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:51.888512    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" event={"ID":"8bc10682-71a0-4e2b-872c-a7dffc915bab","Type":"ContainerDied","Data":"d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6"}
Jul 16 22:46:51.888868 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:51.888572    2556 scope.go:117] "RemoveContainer" containerID="15d8c632a379a57871c572fd4e5ac67bd7229d3bb294f87a48f59105768485a0"
Jul 16 22:46:51.888978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:51.888923    2556 scope.go:117] "RemoveContainer" containerID="d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6"
Jul 16 22:46:51.889167 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:51.889146    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:46:56.129542 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:56.129487    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:56.129946 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:56.129877    2556 scope.go:117] "RemoveContainer" containerID="d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6"
Jul 16 22:46:56.130068 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:56.130049    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:46:56.755545 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:56.755507    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:46:56.913603 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:46:56.913575    2556 scope.go:117] "RemoveContainer" containerID="d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6"
Jul 16 22:46:56.913788 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:46:56.913763    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:47:08.806011 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:08.805980    2556 scope.go:117] "RemoveContainer" containerID="d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6"
Jul 16 22:47:08.806441 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:08.806195    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:47:21.805991 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:21.805955    2556 scope.go:117] "RemoveContainer" containerID="d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6"
Jul 16 22:47:23.041456 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:23.041422    2556 generic.go:356] "Generic (PLEG): container finished" podID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerID="4b155237c47c163026319284eba2f62ca7986a183aede6f996e0415b18c00613" exitCode=1
Jul 16 22:47:23.041892 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:23.041473    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" event={"ID":"8bc10682-71a0-4e2b-872c-a7dffc915bab","Type":"ContainerDied","Data":"4b155237c47c163026319284eba2f62ca7986a183aede6f996e0415b18c00613"}
Jul 16 22:47:23.041892 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:23.041525    2556 scope.go:117] "RemoveContainer" containerID="d9e6b8ca263dd0d6caf091e28b17e0da04bcc25a7470a7c8c8c7a01c6d8445e6"
Jul 16 22:47:23.041892 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:23.041847    2556 scope.go:117] "RemoveContainer" containerID="4b155237c47c163026319284eba2f62ca7986a183aede6f996e0415b18c00613"
Jul 16 22:47:23.042089 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:23.042069    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:47:24.858892 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:24.858850    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:26.129944 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:26.129906    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:47:26.130375 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:26.130315    2556 scope.go:117] "RemoveContainer" containerID="4b155237c47c163026319284eba2f62ca7986a183aede6f996e0415b18c00613"
Jul 16 22:47:26.130508 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:26.130487    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:47:26.756332 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:26.756293    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:47:27.063180 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:27.063151    2556 scope.go:117] "RemoveContainer" containerID="4b155237c47c163026319284eba2f62ca7986a183aede6f996e0415b18c00613"
Jul 16 22:47:27.063429 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:27.063330    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4_odh-ai-gateway-infra(8bc10682-71a0-4e2b-872c-a7dffc915bab)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab"
Jul 16 22:47:29.664990 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:29.664954    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:29.764688 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:29.764651    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:35.520599 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.519951    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"]
Jul 16 22:47:35.552888 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.552803    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:35.664325 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.664298    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:47:35.778648 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.778572    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls\") pod \"8bc10682-71a0-4e2b-872c-a7dffc915bab\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") "
Jul 16 22:47:35.778648 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.778631    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swmhc\" (UniqueName: \"kubernetes.io/projected/8bc10682-71a0-4e2b-872c-a7dffc915bab-kube-api-access-swmhc\") pod \"8bc10682-71a0-4e2b-872c-a7dffc915bab\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") "
Jul 16 22:47:35.778837 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.778725    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc10682-71a0-4e2b-872c-a7dffc915bab-openshift-service-ca\") pod \"8bc10682-71a0-4e2b-872c-a7dffc915bab\" (UID: \"8bc10682-71a0-4e2b-872c-a7dffc915bab\") "
Jul 16 22:47:35.779094 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.779061    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bc10682-71a0-4e2b-872c-a7dffc915bab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8bc10682-71a0-4e2b-872c-a7dffc915bab" (UID: "8bc10682-71a0-4e2b-872c-a7dffc915bab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:47:35.780989 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.780954    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8bc10682-71a0-4e2b-872c-a7dffc915bab" (UID: "8bc10682-71a0-4e2b-872c-a7dffc915bab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:47:35.780989 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.780961    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bc10682-71a0-4e2b-872c-a7dffc915bab-kube-api-access-swmhc" (OuterVolumeSpecName: "kube-api-access-swmhc") pod "8bc10682-71a0-4e2b-872c-a7dffc915bab" (UID: "8bc10682-71a0-4e2b-872c-a7dffc915bab"). InnerVolumeSpecName "kube-api-access-swmhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:47:35.880419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.880380    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bc10682-71a0-4e2b-872c-a7dffc915bab-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.880419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.880412    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bc10682-71a0-4e2b-872c-a7dffc915bab-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:47:35.880419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:35.880426    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swmhc\" (UniqueName: \"kubernetes.io/projected/8bc10682-71a0-4e2b-872c-a7dffc915bab-kube-api-access-swmhc\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:47:36.054362 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:36.054327    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:36.110947 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:36.110916    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"
Jul 16 22:47:36.111148 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:36.110914    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4" event={"ID":"8bc10682-71a0-4e2b-872c-a7dffc915bab","Type":"ContainerDied","Data":"0473e51ff22f03a978cd566c76efc1dbeb22286dd0c36ef0b4a85779c7dd6838"}
Jul 16 22:47:36.111148 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:36.111053    2556 scope.go:117] "RemoveContainer" containerID="4b155237c47c163026319284eba2f62ca7986a183aede6f996e0415b18c00613"
Jul 16 22:47:36.129057 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:36.129031    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"]
Jul 16 22:47:36.133443 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:36.133409    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bf38b381-6fd76b445b-g5mp4"]
Jul 16 22:47:36.243490 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:36.243460    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:37.812935 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:37.812898    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" path="/var/lib/kubelet/pods/8bc10682-71a0-4e2b-872c-a7dffc915bab/volumes"
Jul 16 22:47:41.744181 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:41.744149    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:41.844155 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:41.844072    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:41.951498 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:41.951460    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:42.051790 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:42.051754    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:43.157390 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.157358    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:43.254809 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.254759    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:43.388006 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.387968    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"]
Jul 16 22:47:43.388432 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388415    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.388432 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388434    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.388541 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388452    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.388541 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388457    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.388541 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388466    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:47:43.388541 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388472    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:47:43.388541 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388488    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:47:43.388541 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388493    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:47:43.388727 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388559    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.388727 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388569    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="466122f1-7446-46a2-adec-b1988cbe5974" containerName="maas-api"
Jul 16 22:47:43.388727 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388577    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.388727 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388583    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.388727 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.388588    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:47:43.392684 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.392665    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.395162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.395140    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-234edc78-serving-cert\""
Jul 16 22:47:43.395505 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.395489    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 22:47:43.404851 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.404820    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"]
Jul 16 22:47:43.427901 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.427767    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:43.456655 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.456625    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525231f-2ab7-4650-b6be-49a8bdf66416-openshift-service-ca\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.456773 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.456671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.456773 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.456756    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pft2n\" (UniqueName: \"kubernetes.io/projected/0525231f-2ab7-4650-b6be-49a8bdf66416-kube-api-access-pft2n\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.557789 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.557749    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.557971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.557814    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pft2n\" (UniqueName: \"kubernetes.io/projected/0525231f-2ab7-4650-b6be-49a8bdf66416-kube-api-access-pft2n\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.557971 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:43.557897    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-serving-cert: secret "maas-api-e2e-ait-234edc78-serving-cert" not found
Jul 16 22:47:43.557971 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.557906    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525231f-2ab7-4650-b6be-49a8bdf66416-openshift-service-ca\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.558100 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:43.557984    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls podName:0525231f-2ab7-4650-b6be-49a8bdf66416 nodeName:}" failed. No retries permitted until 2026-07-16 22:47:44.05796227 +0000 UTC m=+1856.731221269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls") pod "maas-api-e2e-ait-234edc78-fc748b778-5528c" (UID: "0525231f-2ab7-4650-b6be-49a8bdf66416") : secret "maas-api-e2e-ait-234edc78-serving-cert" not found
Jul 16 22:47:43.558604 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.558585    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525231f-2ab7-4650-b6be-49a8bdf66416-openshift-service-ca\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.566943 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.566918    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pft2n\" (UniqueName: \"kubernetes.io/projected/0525231f-2ab7-4650-b6be-49a8bdf66416-kube-api-access-pft2n\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:43.814254 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.814215    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:43.919407 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:43.919373    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:44.017087 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:44.017049    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:44.063446 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:44.063405    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:44.066079 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:44.066027    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls\") pod \"maas-api-e2e-ait-234edc78-fc748b778-5528c\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:44.303797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:44.303765    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:44.464666 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:44.464636    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"]
Jul 16 22:47:44.466315 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:47:44.466287    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0525231f_2ab7_4650_b6be_49a8bdf66416.slice/crio-9d7b4fc963a04f514ae56358f51f6e7578b1920c0fc4c9e9769d41256d4ac319 WatchSource:0}: Error finding container 9d7b4fc963a04f514ae56358f51f6e7578b1920c0fc4c9e9769d41256d4ac319: Status 404 returned error can't find the container with id 9d7b4fc963a04f514ae56358f51f6e7578b1920c0fc4c9e9769d41256d4ac319
Jul 16 22:47:45.161617 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:45.161581    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerStarted","Data":"c2275bb200a1864ad6492d1de80cc7e7f372c9b26297ad693171f7cb4cdbfe55"}
Jul 16 22:47:45.161807 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:45.161647    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerStarted","Data":"9d7b4fc963a04f514ae56358f51f6e7578b1920c0fc4c9e9769d41256d4ac319"}
Jul 16 22:47:45.161807 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:45.161774    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:45.181040 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:45.180989    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podStartSLOduration=1.668248981 podStartE2EDuration="2.180974281s" totalImagesPullingTime="512.7253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:47:45.179577081 +0000 UTC m=+1857.852836108" watchObservedRunningTime="2026-07-16 22:47:45.180974281 +0000 UTC m=+1857.854233289"
Jul 16 22:47:45.211228 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:45.211191    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:46.168122 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:46.168076    2556 generic.go:356] "Generic (PLEG): container finished" podID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerID="c2275bb200a1864ad6492d1de80cc7e7f372c9b26297ad693171f7cb4cdbfe55" exitCode=1
Jul 16 22:47:46.168523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:46.168145    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"c2275bb200a1864ad6492d1de80cc7e7f372c9b26297ad693171f7cb4cdbfe55"}
Jul 16 22:47:46.168523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:46.168417    2556 scope.go:117] "RemoveContainer" containerID="c2275bb200a1864ad6492d1de80cc7e7f372c9b26297ad693171f7cb4cdbfe55"
Jul 16 22:47:47.175092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:47.175001    2556 generic.go:356] "Generic (PLEG): container finished" podID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerID="35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942" exitCode=1
Jul 16 22:47:47.175092 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:47.175051    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942"}
Jul 16 22:47:47.175561 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:47.175149    2556 scope.go:117] "RemoveContainer" containerID="c2275bb200a1864ad6492d1de80cc7e7f372c9b26297ad693171f7cb4cdbfe55"
Jul 16 22:47:47.175561 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:47.175430    2556 scope.go:117] "RemoveContainer" containerID="35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942"
Jul 16 22:47:47.175652 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:47.175635    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:47:48.181956 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:48.181924    2556 scope.go:117] "RemoveContainer" containerID="35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942"
Jul 16 22:47:48.182351 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:48.182135    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:47:50.161917 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:50.161874    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:50.162388 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:50.162333    2556 scope.go:117] "RemoveContainer" containerID="35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942"
Jul 16 22:47:50.162569 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:50.162546    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:47:54.304051 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:54.304019    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:47:54.304481 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:47:54.304434    2556 scope.go:117] "RemoveContainer" containerID="35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942"
Jul 16 22:47:54.304659 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:47:54.304637    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:48:05.806271 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:05.806243    2556 scope.go:117] "RemoveContainer" containerID="35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942"
Jul 16 22:48:06.275597 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:06.275563    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerStarted","Data":"569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3"}
Jul 16 22:48:06.275787 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:06.275771    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:48:07.288801 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:07.288762    2556 generic.go:356] "Generic (PLEG): container finished" podID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerID="569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3" exitCode=1
Jul 16 22:48:07.289230 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:07.288823    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3"}
Jul 16 22:48:07.289230 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:07.288909    2556 scope.go:117] "RemoveContainer" containerID="35c97334eb1ac61e359c0fbb96a47dbd3d790052710e364393d5f68311794942"
Jul 16 22:48:07.289230 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:07.289165    2556 scope.go:117] "RemoveContainer" containerID="569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3"
Jul 16 22:48:07.289411 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:48:07.289357    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:48:08.295550 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:08.295522    2556 scope.go:117] "RemoveContainer" containerID="569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3"
Jul 16 22:48:08.295944 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:48:08.295697    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:48:14.304085 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:14.304047    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:48:14.304517 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:14.304502    2556 scope.go:117] "RemoveContainer" containerID="569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3"
Jul 16 22:48:14.304710 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:48:14.304691    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:48:26.805972 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:26.805937    2556 scope.go:117] "RemoveContainer" containerID="569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3"
Jul 16 22:48:26.806428 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:26.806295    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:48:27.392793 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:27.392744    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerStarted","Data":"b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"}
Jul 16 22:48:27.393858 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:27.393786    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:48:28.400358 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:28.400314    2556 generic.go:356] "Generic (PLEG): container finished" podID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436" exitCode=1
Jul 16 22:48:28.400791 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:28.400401    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"}
Jul 16 22:48:28.400791 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:28.400469    2556 scope.go:117] "RemoveContainer" containerID="569a7afb47dcd463fdd87d96f16655540e2d47c85c440aa986c934d0560adec3"
Jul 16 22:48:28.400791 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:28.400712    2556 scope.go:117] "RemoveContainer" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"
Jul 16 22:48:28.400950 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:48:28.400932    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:48:29.408423 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:29.408388    2556 scope.go:117] "RemoveContainer" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"
Jul 16 22:48:29.408911 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:48:29.408586    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:48:34.304693 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:34.304653    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:48:34.307044 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:34.305066    2556 scope.go:117] "RemoveContainer" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"
Jul 16 22:48:34.307044 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:48:34.305282    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:48:49.805835 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:48:49.805800    2556 scope.go:117] "RemoveContainer" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"
Jul 16 22:48:49.806435 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:48:49.806014    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:49:02.806504 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:02.806467    2556 scope.go:117] "RemoveContainer" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"
Jul 16 22:49:02.806996 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:49:02.806668    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:49:14.806053 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:14.806020    2556 scope.go:117] "RemoveContainer" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"
Jul 16 22:49:15.640646 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:15.640610    2556 generic.go:356] "Generic (PLEG): container finished" podID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451" exitCode=1
Jul 16 22:49:15.640835 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:15.640683    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"}
Jul 16 22:49:15.640835 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:15.640743    2556 scope.go:117] "RemoveContainer" containerID="b380dae3b28458443042de3340e98ead518fecc2151693369bbf0126f2f3f436"
Jul 16 22:49:15.641095 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:15.641070    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:49:15.641337 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:49:15.641315    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:49:17.394775 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:17.394736    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:49:17.395304 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:17.395170    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:49:17.395425 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:49:17.395401    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:49:24.304174 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:24.304138    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:49:24.304591 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:24.304572    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:49:24.304779 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:49:24.304756    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:49:38.805543 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:38.805469    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:49:38.806008 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:49:38.805653    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:49:50.805718 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:50.805685    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:49:50.806097 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:49:50.805859    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:49:53.806914 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:53.806878    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:55.392166 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:49:55.392095    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:04.806313 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:04.806282    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:50:04.806713 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:50:04.806449    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:50:17.808511 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:17.808458    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:50:17.809064 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:50:17.808654    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:50:28.806068 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:28.806038    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:50:28.806475 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:50:28.806251    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:50:39.811722 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:39.811695    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:50:41.067691 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:41.067652    2556 generic.go:356] "Generic (PLEG): container finished" podID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67" exitCode=1
Jul 16 22:50:41.068190 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:41.067724    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"}
Jul 16 22:50:41.068190 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:41.067782    2556 scope.go:117] "RemoveContainer" containerID="cabccf8d3d529792a02566839735dc6fb3d620f5d3b16e63813513cf90f48451"
Jul 16 22:50:41.068190 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:41.068149    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:50:41.068425 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:50:41.068405    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:50:42.394290 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:42.394261    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:50:42.394733 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:42.394587    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:50:42.394785 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:50:42.394737    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:50:44.304053 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:44.304017    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:50:44.304451 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:44.304422    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:50:44.304642 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:50:44.304620    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:50:54.806346 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:50:54.806315    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:50:54.806816 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:50:54.806499    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:51:06.805994 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:06.805964    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:51:06.806421 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:51:06.806178    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:51:10.868104 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:10.868023    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:10.969212 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:10.969175    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:19.105290 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:19.105248    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:19.204102 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:19.204065    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:19.806056 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:19.806026    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:51:19.806259 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:51:19.806231    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:51:32.806012 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:32.805980    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:51:32.806501 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:51:32.806204    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:51:47.809692 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:47.809661    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:51:47.810301 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:51:47.809861    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:51:47.918776 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:47.918744    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:51:47.922537 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:47.922510    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:51:47.923733 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:47.923710    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:51:47.927872 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:47.927849    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:51:58.806310 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:51:58.806283    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:51:58.806706 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:51:58.806477    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:52:12.806459 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:52:12.806429    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:52:12.806949 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:52:12.806671    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:52:26.806272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:52:26.806235    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:52:26.806769 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:52:26.806420    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:52:41.806462 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:52:41.806379    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:52:41.806862 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:52:41.806562    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:52:56.806678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:52:56.806641    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:52:56.807089 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:52:56.806866    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:53:08.806564 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:08.806530    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:53:08.807039 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:53:08.806695    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:53:20.805951 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:20.805921    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:53:21.886500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:21.886406    2556 generic.go:356] "Generic (PLEG): container finished" podID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerID="ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe" exitCode=1
Jul 16 22:53:21.886500 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:21.886478    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe"}
Jul 16 22:53:21.887048 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:21.886538    2556 scope.go:117] "RemoveContainer" containerID="12b3429c53b3c2ecc46896c5f35fbfdef7bb8517df4df8898984e98170f4ae67"
Jul 16 22:53:21.887048 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:21.886929    2556 scope.go:117] "RemoveContainer" containerID="ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe"
Jul 16 22:53:21.887216 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:53:21.887194    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:53:22.394205 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:22.394172    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:53:22.897025 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:22.896995    2556 scope.go:117] "RemoveContainer" containerID="ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe"
Jul 16 22:53:22.897441 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:53:22.897238    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:53:24.304279 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:24.304250    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:53:24.304660 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:24.304613    2556 scope.go:117] "RemoveContainer" containerID="ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe"
Jul 16 22:53:24.304798 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:53:24.304776    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:53:34.806606 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:34.806565    2556 scope.go:117] "RemoveContainer" containerID="ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe"
Jul 16 22:53:34.808906 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:53:34.806730    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:53:47.812720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:47.812693    2556 scope.go:117] "RemoveContainer" containerID="ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe"
Jul 16 22:53:47.813217 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:53:47.812876    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-234edc78-fc748b778-5528c_odh-ai-gateway-infra(0525231f-2ab7-4650-b6be-49a8bdf66416)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416"
Jul 16 22:53:48.460887 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:48.460834    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:48.535695 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:48.535666    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:54.470338 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.470302    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"]
Jul 16 22:53:54.500782 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.500748    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:54.611289 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.611261    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:53:54.611834 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.611812    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:54.711857 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.711823    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525231f-2ab7-4650-b6be-49a8bdf66416-openshift-service-ca\") pod \"0525231f-2ab7-4650-b6be-49a8bdf66416\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") "
Jul 16 22:53:54.712024 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.711880    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pft2n\" (UniqueName: \"kubernetes.io/projected/0525231f-2ab7-4650-b6be-49a8bdf66416-kube-api-access-pft2n\") pod \"0525231f-2ab7-4650-b6be-49a8bdf66416\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") "
Jul 16 22:53:54.712024 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.711905    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls\") pod \"0525231f-2ab7-4650-b6be-49a8bdf66416\" (UID: \"0525231f-2ab7-4650-b6be-49a8bdf66416\") "
Jul 16 22:53:54.712214 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.712193    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0525231f-2ab7-4650-b6be-49a8bdf66416-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0525231f-2ab7-4650-b6be-49a8bdf66416" (UID: "0525231f-2ab7-4650-b6be-49a8bdf66416"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:53:54.714158 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.714104    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0525231f-2ab7-4650-b6be-49a8bdf66416-kube-api-access-pft2n" (OuterVolumeSpecName: "kube-api-access-pft2n") pod "0525231f-2ab7-4650-b6be-49a8bdf66416" (UID: "0525231f-2ab7-4650-b6be-49a8bdf66416"). InnerVolumeSpecName "kube-api-access-pft2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:53:54.714158 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.714142    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0525231f-2ab7-4650-b6be-49a8bdf66416" (UID: "0525231f-2ab7-4650-b6be-49a8bdf66416"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:53:54.812929 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.812897    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pft2n\" (UniqueName: \"kubernetes.io/projected/0525231f-2ab7-4650-b6be-49a8bdf66416-kube-api-access-pft2n\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:53:54.812929 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.812924    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0525231f-2ab7-4650-b6be-49a8bdf66416-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:53:54.812929 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:54.812933    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525231f-2ab7-4650-b6be-49a8bdf66416-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:53:55.057709 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:55.057677    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"
Jul 16 22:53:55.057875 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:55.057676    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c" event={"ID":"0525231f-2ab7-4650-b6be-49a8bdf66416","Type":"ContainerDied","Data":"9d7b4fc963a04f514ae56358f51f6e7578b1920c0fc4c9e9769d41256d4ac319"}
Jul 16 22:53:55.057875 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:55.057815    2556 scope.go:117] "RemoveContainer" containerID="ea9cfcf5bfafc95629a99b2653f3fafb5d6eab6ff66b771f4cf6ee130c0f5ffe"
Jul 16 22:53:55.079845 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:55.079819    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"]
Jul 16 22:53:55.085676 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:55.085650    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-234edc78-fc748b778-5528c"]
Jul 16 22:53:55.812004 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:53:55.811968    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" path="/var/lib/kubelet/pods/0525231f-2ab7-4650-b6be-49a8bdf66416/volumes"
Jul 16 22:54:00.796961 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:00.796926    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:00.864795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:00.864765    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:01.116941 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:01.116851    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:02.248844 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.248815    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:02.338648 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.338613    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"]
Jul 16 22:54:02.339514 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339486    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339622 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339517    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339622 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339530    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339622 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339538    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339622 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339566    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339622 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339574    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339584    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339641    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339654    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339663    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339686    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339695    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339707    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339716    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bc10682-71a0-4e2b-872c-a7dffc915bab" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339727    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339735    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339843    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339854    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339864    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.339871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339871    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.340523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.339880    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:54:02.343898 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.343858    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:02.344866 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.344843    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.347707 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.347685    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-75e8dc20-serving-cert\""
Jul 16 22:54:02.347707 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.347711    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 22:54:02.356594 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.356530    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"]
Jul 16 22:54:02.437641 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.437596    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:02.484512 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.484475    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-openshift-service-ca\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.484702 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.484520    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.484702 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.484538    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtqkt\" (UniqueName: \"kubernetes.io/projected/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-kube-api-access-xtqkt\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.585474 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.585439    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-openshift-service-ca\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.585689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.585493    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.585689 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.585512    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtqkt\" (UniqueName: \"kubernetes.io/projected/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-kube-api-access-xtqkt\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.585689 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:02.585613    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-serving-cert: secret "maas-api-e2e-derive-75e8dc20-serving-cert" not found
Jul 16 22:54:02.585860 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:02.585711    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls podName:8aa5121e-54cc-47ef-a3ec-fe3bc69e6745 nodeName:}" failed. No retries permitted until 2026-07-16 22:54:03.085688926 +0000 UTC m=+2235.758947928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls") pod "maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" (UID: "8aa5121e-54cc-47ef-a3ec-fe3bc69e6745") : secret "maas-api-e2e-derive-75e8dc20-serving-cert" not found
Jul 16 22:54:02.586087 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.586068    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-openshift-service-ca\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.606582 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.606546    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtqkt\" (UniqueName: \"kubernetes.io/projected/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-kube-api-access-xtqkt\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:02.812423 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:02.812390    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:03.070744 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:03.070711    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:03.090801 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:03.090765    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:03.093419 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:03.093389    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls\") pod \"maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:03.157741 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:03.157709    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:03.259819 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:03.259786    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:03.414804 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:03.414772    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"]
Jul 16 22:54:03.417049 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:54:03.417024    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa5121e_54cc_47ef_a3ec_fe3bc69e6745.slice/crio-b36f015a22aa643ece93c4e92e70d3861938e96fa8fdba377e7bd0507afa92be WatchSource:0}: Error finding container b36f015a22aa643ece93c4e92e70d3861938e96fa8fdba377e7bd0507afa92be: Status 404 returned error can't find the container with id b36f015a22aa643ece93c4e92e70d3861938e96fa8fdba377e7bd0507afa92be
Jul 16 22:54:03.418814 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:03.418796    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:54:04.105553 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:04.105519    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerStarted","Data":"c099f55a8172c6303b99de2331a490311bb45e443a698408f9d995d2fac284e3"}
Jul 16 22:54:04.105725 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:04.105569    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerStarted","Data":"b36f015a22aa643ece93c4e92e70d3861938e96fa8fdba377e7bd0507afa92be"}
Jul 16 22:54:04.105725 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:04.105655    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:04.132546 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:04.132488    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podStartSLOduration=1.661953072 podStartE2EDuration="2.132467754s" totalImagesPullingTime="470.514682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:54:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:54:04.130722834 +0000 UTC m=+2236.803981864" watchObservedRunningTime="2026-07-16 22:54:04.132467754 +0000 UTC m=+2236.805726762"
Jul 16 22:54:05.112599 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:05.112555    2556 generic.go:356] "Generic (PLEG): container finished" podID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerID="c099f55a8172c6303b99de2331a490311bb45e443a698408f9d995d2fac284e3" exitCode=1
Jul 16 22:54:05.113073 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:05.112639    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerDied","Data":"c099f55a8172c6303b99de2331a490311bb45e443a698408f9d995d2fac284e3"}
Jul 16 22:54:05.113073 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:05.112932    2556 scope.go:117] "RemoveContainer" containerID="c099f55a8172c6303b99de2331a490311bb45e443a698408f9d995d2fac284e3"
Jul 16 22:54:06.119199 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:06.119167    2556 generic.go:356] "Generic (PLEG): container finished" podID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerID="25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44" exitCode=1
Jul 16 22:54:06.119654 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:06.119248    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerDied","Data":"25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44"}
Jul 16 22:54:06.119654 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:06.119312    2556 scope.go:117] "RemoveContainer" containerID="c099f55a8172c6303b99de2331a490311bb45e443a698408f9d995d2fac284e3"
Jul 16 22:54:06.119654 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:06.119517    2556 scope.go:117] "RemoveContainer" containerID="25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44"
Jul 16 22:54:06.119773 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:06.119706    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:07.127864 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:07.127822    2556 scope.go:117] "RemoveContainer" containerID="25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44"
Jul 16 22:54:07.128285 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:07.128012    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:09.106781 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:09.106688    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:09.107209 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:09.107077    2556 scope.go:117] "RemoveContainer" containerID="25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44"
Jul 16 22:54:09.107314 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:09.107294    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:13.260506 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:13.260461    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:13.260918 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:13.260880    2556 scope.go:117] "RemoveContainer" containerID="25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44"
Jul 16 22:54:13.261089 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:13.261070    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:24.991845 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:24.991805    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:25.092004 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:25.091967    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:27.812720 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:27.812693    2556 scope.go:117] "RemoveContainer" containerID="25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44"
Jul 16 22:54:28.664813 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:28.664770    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:29.241647 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:29.241615    2556 generic.go:356] "Generic (PLEG): container finished" podID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerID="a5b4664c04d52bb559d24ada7671ec53b580c7da30c57a9a2dc53f74a1048caa" exitCode=1
Jul 16 22:54:29.242097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:29.241691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerDied","Data":"a5b4664c04d52bb559d24ada7671ec53b580c7da30c57a9a2dc53f74a1048caa"}
Jul 16 22:54:29.242097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:29.241751    2556 scope.go:117] "RemoveContainer" containerID="25b8935bfad1420a253a1efd6557808dd2276dc037a65a9b47a731f47a90af44"
Jul 16 22:54:29.242097 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:29.242067    2556 scope.go:117] "RemoveContainer" containerID="a5b4664c04d52bb559d24ada7671ec53b580c7da30c57a9a2dc53f74a1048caa"
Jul 16 22:54:29.242333 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:29.242313    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:33.260162 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:33.260103    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:33.260659 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:33.260527    2556 scope.go:117] "RemoveContainer" containerID="a5b4664c04d52bb559d24ada7671ec53b580c7da30c57a9a2dc53f74a1048caa"
Jul 16 22:54:33.260710 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:33.260683    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:34.106303 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:34.106264    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:34.106656 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:34.106641    2556 scope.go:117] "RemoveContainer" containerID="a5b4664c04d52bb559d24ada7671ec53b580c7da30c57a9a2dc53f74a1048caa"
Jul 16 22:54:34.106861 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:34.106841    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:48.806394 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:48.806357    2556 scope.go:117] "RemoveContainer" containerID="a5b4664c04d52bb559d24ada7671ec53b580c7da30c57a9a2dc53f74a1048caa"
Jul 16 22:54:49.349746 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:49.349662    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerStarted","Data":"5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"}
Jul 16 22:54:49.349897 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:49.349862    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:50.356994 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:50.356955    2556 generic.go:356] "Generic (PLEG): container finished" podID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0" exitCode=1
Jul 16 22:54:50.357497 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:50.357030    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerDied","Data":"5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"}
Jul 16 22:54:50.357497 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:50.357084    2556 scope.go:117] "RemoveContainer" containerID="a5b4664c04d52bb559d24ada7671ec53b580c7da30c57a9a2dc53f74a1048caa"
Jul 16 22:54:50.357497 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:50.357304    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:54:50.357661 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:50.357528    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:51.363773 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:51.363743    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:54:51.364184 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:51.363942    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:54:53.260980 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:53.260943    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:54:53.261403 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:54:53.261373    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:54:53.261583 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:54:53.261562    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:55:04.805678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:04.805646    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:55:04.806183 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:55:04.805881    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:55:15.806477 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:15.806440    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:55:15.806939 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:55:15.806696    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:55:28.806688 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:28.806661    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:55:28.807075 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:55:28.806793    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:55:36.246468 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:36.246431    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:40.806032 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:40.805950    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:55:41.619772 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:41.619735    2556 generic.go:356] "Generic (PLEG): container finished" podID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a" exitCode=1
Jul 16 22:55:41.619928 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:41.619804    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerDied","Data":"014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"}
Jul 16 22:55:41.619928 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:41.619864    2556 scope.go:117] "RemoveContainer" containerID="5de4e58dd8090a77bd63b119e7253364de16b899131e78dfa42dbfba8e47b8e0"
Jul 16 22:55:41.620272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:41.620250    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:55:41.620507 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:55:41.620486    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:55:43.260888 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:43.260851    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:55:43.261367 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:43.261219    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:55:43.261425 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:55:43.261390    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:55:44.350050 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:44.350016    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:55:44.350547 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:44.350488    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:55:44.350721 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:55:44.350697    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:55:55.813237 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:55:55.813201    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:55:55.813694 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:55:55.813552    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:56:09.812478 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:09.812448    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:56:09.812954 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:56:09.812623    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:56:11.770085 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:11.770050    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:11.870490 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:11.870440    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:17.702041 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:17.701999    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:17.826559 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:17.826510    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:21.275941 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:21.275907    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:23.806486 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:23.806447    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:56:23.806877 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:56:23.806700    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:56:33.715267 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:33.715227    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:35.806396 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:35.806354    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:56:35.806840 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:56:35.806603    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:56:47.950077 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:47.950051    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:56:47.954631 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:47.954612    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:56:47.955992 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:47.955962    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 22:56:47.960087 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:47.960068    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 22:56:50.805978 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:50.805942    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:56:50.806523 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:56:50.806181    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:56:58.410898 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:56:58.410858    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:01.806327 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:01.806291    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:57:03.022368 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:03.022282    2556 generic.go:356] "Generic (PLEG): container finished" podID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerID="dbeea9b3c133c435c2b772cf42e627f40dde3312374dea937b3972232837eb6a" exitCode=1
Jul 16 22:57:03.022368 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:03.022356    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerDied","Data":"dbeea9b3c133c435c2b772cf42e627f40dde3312374dea937b3972232837eb6a"}
Jul 16 22:57:03.022792 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:03.022422    2556 scope.go:117] "RemoveContainer" containerID="014eb13006d640545e9ab2e6e6106ea92a6ccfe0f46f5af8e760e6d1079d6f3a"
Jul 16 22:57:03.022792 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:03.022748    2556 scope.go:117] "RemoveContainer" containerID="dbeea9b3c133c435c2b772cf42e627f40dde3312374dea937b3972232837eb6a"
Jul 16 22:57:03.023014 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:57:03.022993    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:57:03.260956 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:03.260922    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:57:04.028996 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:04.028959    2556 scope.go:117] "RemoveContainer" containerID="dbeea9b3c133c435c2b772cf42e627f40dde3312374dea937b3972232837eb6a"
Jul 16 22:57:04.029404 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:57:04.029172    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:57:04.350015 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:04.349980    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:57:05.033642 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:05.033615    2556 scope.go:117] "RemoveContainer" containerID="dbeea9b3c133c435c2b772cf42e627f40dde3312374dea937b3972232837eb6a"
Jul 16 22:57:05.034099 ip-10-0-139-105 kubenswrapper[2556]: E0716 22:57:05.033811    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz_odh-ai-gateway-infra(8aa5121e-54cc-47ef-a3ec-fe3bc69e6745)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"
Jul 16 22:57:06.768412 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:06.768379    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:06.946764 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:06.946730    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:12.665308 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.665273    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"]
Jul 16 22:57:12.694302 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.694270    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:12.809955 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.809929    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:57:12.840560 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.840527    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-openshift-service-ca\") pod \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") "
Jul 16 22:57:12.840699 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.840624    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtqkt\" (UniqueName: \"kubernetes.io/projected/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-kube-api-access-xtqkt\") pod \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") "
Jul 16 22:57:12.840699 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.840669    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls\") pod \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\" (UID: \"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745\") "
Jul 16 22:57:12.840839 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.840818    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" (UID: "8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:57:12.840930 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.840917    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.842890 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.842859    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" (UID: "8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:57:12.843013 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.842944    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-kube-api-access-xtqkt" (OuterVolumeSpecName: "kube-api-access-xtqkt") pod "8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" (UID: "8aa5121e-54cc-47ef-a3ec-fe3bc69e6745"). InnerVolumeSpecName "kube-api-access-xtqkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:57:12.941637 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.941559    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtqkt\" (UniqueName: \"kubernetes.io/projected/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-kube-api-access-xtqkt\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:57:12.941637 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:12.941588    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:57:13.082289 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:13.082252    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz" event={"ID":"8aa5121e-54cc-47ef-a3ec-fe3bc69e6745","Type":"ContainerDied","Data":"b36f015a22aa643ece93c4e92e70d3861938e96fa8fdba377e7bd0507afa92be"}
Jul 16 22:57:13.082289 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:13.082280    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"
Jul 16 22:57:13.082577 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:13.082317    2556 scope.go:117] "RemoveContainer" containerID="dbeea9b3c133c435c2b772cf42e627f40dde3312374dea937b3972232837eb6a"
Jul 16 22:57:13.082771 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:13.082751    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:13.105728 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:13.105704    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"]
Jul 16 22:57:13.113085 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:13.113061    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-75e8dc20-db4dccb95-bnnkz"]
Jul 16 22:57:13.812309 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:13.812280    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" path="/var/lib/kubelet/pods/8aa5121e-54cc-47ef-a3ec-fe3bc69e6745/volumes"
Jul 16 22:57:18.691673 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:18.691641    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:18.793366 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:18.793334    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:19.256938 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:19.256885    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:19.403832 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:19.403805    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:20.883799 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:20.883761    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:23.505523 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:23.505463    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:26.821304 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:26.821268    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:26.933307 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:26.933273    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:27.025934 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:27.025892    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:27.124144 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:27.124027    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:27.535518 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:27.535431    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:53.483008 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:53.482974    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:59.743815 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:59.743774    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:59.860129 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:59.860082    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:59.957958 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:57:59.957925    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:15.901589 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:15.901545    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:22.404008 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:22.403969    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:22.515844 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:22.515809    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:22.604951 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:22.604916    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:33.137518 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:33.137484    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:33.930183 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:33.930148    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:39.808913 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:39.808835    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:39.982942 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:39.982909    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:40.080140 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:40.080046    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:40.849564 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:40.849531    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:40.939526 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:40.939489    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:42.522163 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:42.522106    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:48.677836 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:48.677805    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:48.754284 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:48.754257    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:49.333545 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:49.333505    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:55.417930 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:55.417879    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:55.492270 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:55.492229    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:55.676681 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:55.676595    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:55.962465 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:55.962373    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:56.062407 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:56.062375    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:59.187522 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.187472    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:59.291244 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291209    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"]
Jul 16 22:58:59.291701 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291683    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291704    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291714    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291722    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291738    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291746    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291759    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291767    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291787    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.291797 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291795    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291806    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291814    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291928    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291940    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291956    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291964    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291973    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291980    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:58:59.292272 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.291989    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0525231f-2ab7-4650-b6be-49a8bdf66416" containerName="maas-api"
Jul 16 22:58:59.295300 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.295278    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.297866 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.297844    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 22:58:59.297989 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.297844    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bd998308-serving-cert\""
Jul 16 22:58:59.313143 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.313090    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"]
Jul 16 22:58:59.316994 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.316969    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:59.367925 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.367895    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b926292-129a-4ce4-a843-a277babd10c5-openshift-service-ca\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.368051 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.367935    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v669d\" (UniqueName: \"kubernetes.io/projected/5b926292-129a-4ce4-a843-a277babd10c5-kube-api-access-v669d\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.368051 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.368037    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b926292-129a-4ce4-a843-a277babd10c5-maas-api-tls\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.403265 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.403236    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:59.469373 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.469291    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b926292-129a-4ce4-a843-a277babd10c5-openshift-service-ca\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.469373 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.469330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v669d\" (UniqueName: \"kubernetes.io/projected/5b926292-129a-4ce4-a843-a277babd10c5-kube-api-access-v669d\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.469592 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.469479    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b926292-129a-4ce4-a843-a277babd10c5-maas-api-tls\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.469989 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.469972    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b926292-129a-4ce4-a843-a277babd10c5-openshift-service-ca\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.472050 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.472032    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b926292-129a-4ce4-a843-a277babd10c5-maas-api-tls\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.488940 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.488915    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v669d\" (UniqueName: \"kubernetes.io/projected/5b926292-129a-4ce4-a843-a277babd10c5-kube-api-access-v669d\") pod \"maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.502292 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.502271    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:59.608241 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.608210    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:58:59.758561 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.758481    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"]
Jul 16 22:58:59.760799 ip-10-0-139-105 kubenswrapper[2556]: W0716 22:58:59.760771    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b926292_129a_4ce4_a843_a277babd10c5.slice/crio-6684579a4a2a9aa475b6d4e3601a9f3ef71cb33096311cf0328442d91b3d27d3 WatchSource:0}: Error finding container 6684579a4a2a9aa475b6d4e3601a9f3ef71cb33096311cf0328442d91b3d27d3: Status 404 returned error can't find the container with id 6684579a4a2a9aa475b6d4e3601a9f3ef71cb33096311cf0328442d91b3d27d3
Jul 16 22:58:59.922970 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:58:59.922936    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:00.022160 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:00.022058    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:00.625305 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:00.625263    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh" event={"ID":"5b926292-129a-4ce4-a843-a277babd10c5","Type":"ContainerStarted","Data":"9e3399300c786a0bd89573390f72ead6a090f69da5679ab8ff791241196cc0cd"}
Jul 16 22:59:00.625735 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:00.625358    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh" event={"ID":"5b926292-129a-4ce4-a843-a277babd10c5","Type":"ContainerStarted","Data":"6684579a4a2a9aa475b6d4e3601a9f3ef71cb33096311cf0328442d91b3d27d3"}
Jul 16 22:59:00.625735 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:00.625474    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:59:00.647632 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:00.647582    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh" podStartSLOduration=1.266477476 podStartE2EDuration="1.647567676s" totalImagesPullingTime="381.0902ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:59:00.646548723 +0000 UTC m=+2533.319807731" watchObservedRunningTime="2026-07-16 22:59:00.647567676 +0000 UTC m=+2533.320826683"
Jul 16 22:59:03.572712 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:03.572677    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:04.213799 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:04.213741    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:04.304800 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:04.304770    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:04.620830 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:04.620797    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:06.636463 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:06.636421    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:59:10.613914 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:10.613878    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:10.715353 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:10.715309    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:21.484626 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.484592    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"]
Jul 16 22:59:21.485209 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.484874    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh" podUID="5b926292-129a-4ce4-a843-a277babd10c5" containerName="maas-api" containerID="cri-o://9e3399300c786a0bd89573390f72ead6a090f69da5679ab8ff791241196cc0cd" gracePeriod=30
Jul 16 22:59:21.531823 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.531787    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:21.631664 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.631626    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh" podUID="5b926292-129a-4ce4-a843-a277babd10c5" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.57:8443/health\": dial tcp 10.132.0.57:8443: connect: connection refused"
Jul 16 22:59:21.743366 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.743290    2556 generic.go:356] "Generic (PLEG): container finished" podID="5b926292-129a-4ce4-a843-a277babd10c5" containerID="9e3399300c786a0bd89573390f72ead6a090f69da5679ab8ff791241196cc0cd" exitCode=0
Jul 16 22:59:21.743521 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.743364    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh" event={"ID":"5b926292-129a-4ce4-a843-a277babd10c5","Type":"ContainerDied","Data":"9e3399300c786a0bd89573390f72ead6a090f69da5679ab8ff791241196cc0cd"}
Jul 16 22:59:21.743521 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.743429    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh" event={"ID":"5b926292-129a-4ce4-a843-a277babd10c5","Type":"ContainerDied","Data":"6684579a4a2a9aa475b6d4e3601a9f3ef71cb33096311cf0328442d91b3d27d3"}
Jul 16 22:59:21.743521 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.743447    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6684579a4a2a9aa475b6d4e3601a9f3ef71cb33096311cf0328442d91b3d27d3"
Jul 16 22:59:21.744240 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.744225    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:59:21.876171 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.876136    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b926292-129a-4ce4-a843-a277babd10c5-maas-api-tls\") pod \"5b926292-129a-4ce4-a843-a277babd10c5\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") "
Jul 16 22:59:21.876367 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.876208    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b926292-129a-4ce4-a843-a277babd10c5-openshift-service-ca\") pod \"5b926292-129a-4ce4-a843-a277babd10c5\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") "
Jul 16 22:59:21.876367 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.876277    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v669d\" (UniqueName: \"kubernetes.io/projected/5b926292-129a-4ce4-a843-a277babd10c5-kube-api-access-v669d\") pod \"5b926292-129a-4ce4-a843-a277babd10c5\" (UID: \"5b926292-129a-4ce4-a843-a277babd10c5\") "
Jul 16 22:59:21.876626 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.876600    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b926292-129a-4ce4-a843-a277babd10c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b926292-129a-4ce4-a843-a277babd10c5" (UID: "5b926292-129a-4ce4-a843-a277babd10c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:59:21.878458 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.878435    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b926292-129a-4ce4-a843-a277babd10c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b926292-129a-4ce4-a843-a277babd10c5" (UID: "5b926292-129a-4ce4-a843-a277babd10c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:59:21.878526 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.878485    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b926292-129a-4ce4-a843-a277babd10c5-kube-api-access-v669d" (OuterVolumeSpecName: "kube-api-access-v669d") pod "5b926292-129a-4ce4-a843-a277babd10c5" (UID: "5b926292-129a-4ce4-a843-a277babd10c5"). InnerVolumeSpecName "kube-api-access-v669d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:59:21.977895 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.977856    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b926292-129a-4ce4-a843-a277babd10c5-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:59:21.977895 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.977887    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v669d\" (UniqueName: \"kubernetes.io/projected/5b926292-129a-4ce4-a843-a277babd10c5-kube-api-access-v669d\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:59:21.977895 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:21.977896    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b926292-129a-4ce4-a843-a277babd10c5-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 22:59:22.748236 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:22.748203    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"
Jul 16 22:59:22.770388 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:22.770270    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"]
Jul 16 22:59:22.772203 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:22.772178    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd998308-5d8dd7cc8b-jxzxh"]
Jul 16 22:59:23.812333 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:23.812300    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b926292-129a-4ce4-a843-a277babd10c5" path="/var/lib/kubelet/pods/5b926292-129a-4ce4-a843-a277babd10c5/volumes"
Jul 16 22:59:26.098445 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:26.098409    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:32.391697 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:32.391656    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:32.461930 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:32.461888    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:36.753871 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:36.753822    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:37.103981 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:37.103947    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:37.258587 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:37.258540    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:37.437614 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:37.437522    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:40.642533 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:40.642498    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:40.742783 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:40.742752    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:43.219166 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:43.219078    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:43.322586 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:43.322554    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:43.414008 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:43.413971    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:43.548319 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:43.548286    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:49.727721 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:49.727687    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:49.827360 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:49.827328    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:56.426229 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:56.426195    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:56.749482 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:56.749392    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:56.855532 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:56.855501    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:59.491080 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.491044    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:59.566258 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.566222    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:59.689090 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.689056    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"]
Jul 16 22:59:59.689539 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.689517    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:59:59.689539 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.689537    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:59:59.689678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.689560    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b926292-129a-4ce4-a843-a277babd10c5" containerName="maas-api"
Jul 16 22:59:59.689678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.689566    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b926292-129a-4ce4-a843-a277babd10c5" containerName="maas-api"
Jul 16 22:59:59.689678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.689628    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b926292-129a-4ce4-a843-a277babd10c5" containerName="maas-api"
Jul 16 22:59:59.689678 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.689636    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa5121e-54cc-47ef-a3ec-fe3bc69e6745" containerName="maas-api"
Jul 16 22:59:59.692687 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.692670    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.696359 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.696320    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-25067845-serving-cert\""
Jul 16 22:59:59.696479 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.696407    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 22:59:59.706480 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.706455    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"]
Jul 16 22:59:59.727560 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.727527    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:59.820001 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.819974    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/168b83fa-b9ae-46e8-999b-bc60420d32c9-maas-api-tls\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.820168 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.820003    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/168b83fa-b9ae-46e8-999b-bc60420d32c9-openshift-service-ca\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.820168 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.820021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpzzm\" (UniqueName: \"kubernetes.io/projected/168b83fa-b9ae-46e8-999b-bc60420d32c9-kube-api-access-rpzzm\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.921133 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.921082    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/168b83fa-b9ae-46e8-999b-bc60420d32c9-maas-api-tls\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.921269 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.921154    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/168b83fa-b9ae-46e8-999b-bc60420d32c9-openshift-service-ca\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.921269 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.921184    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpzzm\" (UniqueName: \"kubernetes.io/projected/168b83fa-b9ae-46e8-999b-bc60420d32c9-kube-api-access-rpzzm\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.921966 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.921894    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/168b83fa-b9ae-46e8-999b-bc60420d32c9-openshift-service-ca\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.923885 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.923860    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/168b83fa-b9ae-46e8-999b-bc60420d32c9-maas-api-tls\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.930795 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.930739    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpzzm\" (UniqueName: \"kubernetes.io/projected/168b83fa-b9ae-46e8-999b-bc60420d32c9-kube-api-access-rpzzm\") pod \"maas-api-e2e-mt-25067845-85485685b-4gjrp\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 22:59:59.969630 ip-10-0-139-105 kubenswrapper[2556]: I0716 22:59:59.969606    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:00.003753 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.003731    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 23:00:00.158381 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.157850    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"]
Jul 16 23:00:00.164781 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:00:00.164739    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168b83fa_b9ae_46e8_999b_bc60420d32c9.slice/crio-f4726d9518a08a9e18071ea298806ed5647802ea5dfd1b090994042b8a8aba6d WatchSource:0}: Error finding container f4726d9518a08a9e18071ea298806ed5647802ea5dfd1b090994042b8a8aba6d: Status 404 returned error can't find the container with id f4726d9518a08a9e18071ea298806ed5647802ea5dfd1b090994042b8a8aba6d
Jul 16 23:00:00.166409 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.166385    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 23:00:00.393939 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.393844    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:00.505676 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.505638    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:00.941756 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.941651    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp" event={"ID":"168b83fa-b9ae-46e8-999b-bc60420d32c9","Type":"ContainerStarted","Data":"3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea"}
Jul 16 23:00:00.941756 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.941721    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp" event={"ID":"168b83fa-b9ae-46e8-999b-bc60420d32c9","Type":"ContainerStarted","Data":"f4726d9518a08a9e18071ea298806ed5647802ea5dfd1b090994042b8a8aba6d"}
Jul 16 23:00:00.941756 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.941748    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 23:00:00.962502 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:00.962444    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp" podStartSLOduration=1.468075557 podStartE2EDuration="1.962428204s" totalImagesPullingTime="494.352647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:00.961083799 +0000 UTC m=+2593.634342833" watchObservedRunningTime="2026-07-16 23:00:00.962428204 +0000 UTC m=+2593.635687210"
Jul 16 23:00:03.923159 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:03.923105    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:06.953160 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:06.953133    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 23:00:09.615134 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:09.615029    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:10.765049 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:10.765015    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:16.949512 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:16.949473    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"]
Jul 16 23:00:16.953303 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:16.953283    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"
Jul 16 23:00:16.955677 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:16.955660    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:00:16.960318 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:16.960296    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"]
Jul 16 23:00:17.077834 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:17.077806    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d512aa0b-5a03-40c2-84c4-5ba1c55287eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw\" (UID: \"d512aa0b-5a03-40c2-84c4-5ba1c55287eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"
Jul 16 23:00:17.179186 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:17.179160    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d512aa0b-5a03-40c2-84c4-5ba1c55287eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw\" (UID: \"d512aa0b-5a03-40c2-84c4-5ba1c55287eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"
Jul 16 23:00:17.179715 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:17.179694    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d512aa0b-5a03-40c2-84c4-5ba1c55287eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw\" (UID: \"d512aa0b-5a03-40c2-84c4-5ba1c55287eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"
Jul 16 23:00:17.266672 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:17.266600    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"
Jul 16 23:00:17.412156 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:17.412094    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"]
Jul 16 23:00:17.414450 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:00:17.414422    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd512aa0b_5a03_40c2_84c4_5ba1c55287eb.slice/crio-ecd7af0421559bc042e2c70650e2d7257aa23e37f6d3b9f4a3191e456d8863ff WatchSource:0}: Error finding container ecd7af0421559bc042e2c70650e2d7257aa23e37f6d3b9f4a3191e456d8863ff: Status 404 returned error can't find the container with id ecd7af0421559bc042e2c70650e2d7257aa23e37f6d3b9f4a3191e456d8863ff
Jul 16 23:00:18.028439 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:18.028394    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" event={"ID":"d512aa0b-5a03-40c2-84c4-5ba1c55287eb","Type":"ContainerStarted","Data":"ecd7af0421559bc042e2c70650e2d7257aa23e37f6d3b9f4a3191e456d8863ff"}
Jul 16 23:00:20.041230 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:20.041197    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" event={"ID":"d512aa0b-5a03-40c2-84c4-5ba1c55287eb","Type":"ContainerStarted","Data":"8ffddd486e14c4bbb0788e8e10ff1d48265d0d1ec7e5bca5e4cdbc6c120b0652"}
Jul 16 23:00:20.059258 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:20.059212    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" podStartSLOduration=1.757312064 podStartE2EDuration="4.059196213s" totalImagesPullingTime="2.301884149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:20.05861061 +0000 UTC m=+2612.731869619" watchObservedRunningTime="2026-07-16 23:00:20.059196213 +0000 UTC m=+2612.732455221"
Jul 16 23:00:21.048805 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:21.048769    2556 generic.go:356] "Generic (PLEG): container finished" podID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerID="8ffddd486e14c4bbb0788e8e10ff1d48265d0d1ec7e5bca5e4cdbc6c120b0652" exitCode=6
Jul 16 23:00:21.049194 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:21.048843    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" event={"ID":"d512aa0b-5a03-40c2-84c4-5ba1c55287eb","Type":"ContainerDied","Data":"8ffddd486e14c4bbb0788e8e10ff1d48265d0d1ec7e5bca5e4cdbc6c120b0652"}
Jul 16 23:00:21.049194 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:21.048980    2556 scope.go:117] "RemoveContainer" containerID="8ffddd486e14c4bbb0788e8e10ff1d48265d0d1ec7e5bca5e4cdbc6c120b0652"
Jul 16 23:00:22.055867 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:22.055831    2556 generic.go:356] "Generic (PLEG): container finished" podID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerID="7d919d5e63d2f40c76f0bb8fc291d8c816cfd6ac61c98aa965d25efa7c980ead" exitCode=6
Jul 16 23:00:22.056345 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:22.055877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" event={"ID":"d512aa0b-5a03-40c2-84c4-5ba1c55287eb","Type":"ContainerDied","Data":"7d919d5e63d2f40c76f0bb8fc291d8c816cfd6ac61c98aa965d25efa7c980ead"}
Jul 16 23:00:22.056345 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:22.055934    2556 scope.go:117] "RemoveContainer" containerID="8ffddd486e14c4bbb0788e8e10ff1d48265d0d1ec7e5bca5e4cdbc6c120b0652"
Jul 16 23:00:22.056345 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:22.056098    2556 scope.go:117] "RemoveContainer" containerID="7d919d5e63d2f40c76f0bb8fc291d8c816cfd6ac61c98aa965d25efa7c980ead"
Jul 16 23:00:22.056469 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:00:22.056347    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw_odh-ai-gateway-infra(d512aa0b-5a03-40c2-84c4-5ba1c55287eb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb"
Jul 16 23:00:23.063155 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:23.063097    2556 scope.go:117] "RemoveContainer" containerID="7d919d5e63d2f40c76f0bb8fc291d8c816cfd6ac61c98aa965d25efa7c980ead"
Jul 16 23:00:23.063553 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:00:23.063329    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw_odh-ai-gateway-infra(d512aa0b-5a03-40c2-84c4-5ba1c55287eb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb"
Jul 16 23:00:34.806565 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:34.806531    2556 scope.go:117] "RemoveContainer" containerID="7d919d5e63d2f40c76f0bb8fc291d8c816cfd6ac61c98aa965d25efa7c980ead"
Jul 16 23:00:36.135524 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:36.135491    2556 generic.go:356] "Generic (PLEG): container finished" podID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerID="1f0194ede30ee00f749e62c44cb52230b68b71a93119022511fb82fe172fcbc1" exitCode=6
Jul 16 23:00:36.135993 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:36.135567    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" event={"ID":"d512aa0b-5a03-40c2-84c4-5ba1c55287eb","Type":"ContainerDied","Data":"1f0194ede30ee00f749e62c44cb52230b68b71a93119022511fb82fe172fcbc1"}
Jul 16 23:00:36.135993 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:36.135645    2556 scope.go:117] "RemoveContainer" containerID="7d919d5e63d2f40c76f0bb8fc291d8c816cfd6ac61c98aa965d25efa7c980ead"
Jul 16 23:00:36.135993 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:36.135936    2556 scope.go:117] "RemoveContainer" containerID="1f0194ede30ee00f749e62c44cb52230b68b71a93119022511fb82fe172fcbc1"
Jul 16 23:00:36.136239 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:00:36.136211    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw_odh-ai-gateway-infra(d512aa0b-5a03-40c2-84c4-5ba1c55287eb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb"
Jul 16 23:00:37.162453 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.162416    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"]
Jul 16 23:00:37.304488 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.304454    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"
Jul 16 23:00:37.361843 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.361806    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d512aa0b-5a03-40c2-84c4-5ba1c55287eb-maas-api-service-ca\") pod \"d512aa0b-5a03-40c2-84c4-5ba1c55287eb\" (UID: \"d512aa0b-5a03-40c2-84c4-5ba1c55287eb\") "
Jul 16 23:00:37.362187 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.362167    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d512aa0b-5a03-40c2-84c4-5ba1c55287eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d512aa0b-5a03-40c2-84c4-5ba1c55287eb" (UID: "d512aa0b-5a03-40c2-84c4-5ba1c55287eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:00:37.444094 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.444007    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"]
Jul 16 23:00:37.444380 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.444344    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp" podUID="168b83fa-b9ae-46e8-999b-bc60420d32c9" containerName="maas-api" containerID="cri-o://3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea" gracePeriod=30
Jul 16 23:00:37.462580 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.462558    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d512aa0b-5a03-40c2-84c4-5ba1c55287eb-maas-api-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:00:37.481813 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.481787    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:37.722773 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.722745    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 23:00:37.765956 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.765923    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/168b83fa-b9ae-46e8-999b-bc60420d32c9-maas-api-tls\") pod \"168b83fa-b9ae-46e8-999b-bc60420d32c9\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") "
Jul 16 23:00:37.766137 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.766076    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpzzm\" (UniqueName: \"kubernetes.io/projected/168b83fa-b9ae-46e8-999b-bc60420d32c9-kube-api-access-rpzzm\") pod \"168b83fa-b9ae-46e8-999b-bc60420d32c9\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") "
Jul 16 23:00:37.766218 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.766149    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/168b83fa-b9ae-46e8-999b-bc60420d32c9-openshift-service-ca\") pod \"168b83fa-b9ae-46e8-999b-bc60420d32c9\" (UID: \"168b83fa-b9ae-46e8-999b-bc60420d32c9\") "
Jul 16 23:00:37.766443 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.766415    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/168b83fa-b9ae-46e8-999b-bc60420d32c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "168b83fa-b9ae-46e8-999b-bc60420d32c9" (UID: "168b83fa-b9ae-46e8-999b-bc60420d32c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:00:37.768364 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.768338    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/168b83fa-b9ae-46e8-999b-bc60420d32c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "168b83fa-b9ae-46e8-999b-bc60420d32c9" (UID: "168b83fa-b9ae-46e8-999b-bc60420d32c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:00:37.768474 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.768340    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/168b83fa-b9ae-46e8-999b-bc60420d32c9-kube-api-access-rpzzm" (OuterVolumeSpecName: "kube-api-access-rpzzm") pod "168b83fa-b9ae-46e8-999b-bc60420d32c9" (UID: "168b83fa-b9ae-46e8-999b-bc60420d32c9"). InnerVolumeSpecName "kube-api-access-rpzzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:00:37.867097 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.867051    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpzzm\" (UniqueName: \"kubernetes.io/projected/168b83fa-b9ae-46e8-999b-bc60420d32c9-kube-api-access-rpzzm\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:00:37.867097 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.867082    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/168b83fa-b9ae-46e8-999b-bc60420d32c9-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:00:37.867097 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:37.867091    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/168b83fa-b9ae-46e8-999b-bc60420d32c9-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:00:38.148545 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.148516    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"
Jul 16 23:00:38.148753 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.148553    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw" event={"ID":"d512aa0b-5a03-40c2-84c4-5ba1c55287eb","Type":"ContainerDied","Data":"ecd7af0421559bc042e2c70650e2d7257aa23e37f6d3b9f4a3191e456d8863ff"}
Jul 16 23:00:38.148753 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.148619    2556 scope.go:117] "RemoveContainer" containerID="1f0194ede30ee00f749e62c44cb52230b68b71a93119022511fb82fe172fcbc1"
Jul 16 23:00:38.149837 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.149812    2556 generic.go:356] "Generic (PLEG): container finished" podID="168b83fa-b9ae-46e8-999b-bc60420d32c9" containerID="3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea" exitCode=0
Jul 16 23:00:38.149915 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.149852    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"
Jul 16 23:00:38.149963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.149913    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp" event={"ID":"168b83fa-b9ae-46e8-999b-bc60420d32c9","Type":"ContainerDied","Data":"3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea"}
Jul 16 23:00:38.149963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.149953    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp" event={"ID":"168b83fa-b9ae-46e8-999b-bc60420d32c9","Type":"ContainerDied","Data":"f4726d9518a08a9e18071ea298806ed5647802ea5dfd1b090994042b8a8aba6d"}
Jul 16 23:00:38.157533 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.157513    2556 scope.go:117] "RemoveContainer" containerID="3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea"
Jul 16 23:00:38.165258 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.165097    2556 scope.go:117] "RemoveContainer" containerID="3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea"
Jul 16 23:00:38.165487 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:00:38.165362    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea\": container with ID starting with 3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea not found: ID does not exist" containerID="3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea"
Jul 16 23:00:38.165487 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.165392    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea"} err="failed to get container status \"3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea\": rpc error: code = NotFound desc = could not find container \"3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea\": container with ID starting with 3e3f94cc5ccd9cbfe3d439c3d8b913ee9c2f2592b8172e55ffa9a207873919ea not found: ID does not exist"
Jul 16 23:00:38.170331 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.170310    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"]
Jul 16 23:00:38.180522 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.180502    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-25067845-85485685b-4gjrp"]
Jul 16 23:00:38.200215 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.200195    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"]
Jul 16 23:00:38.204105 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:38.204077    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-75e8dc20-45dgw"]
Jul 16 23:00:39.814163 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:39.814128    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="168b83fa-b9ae-46e8-999b-bc60420d32c9" path="/var/lib/kubelet/pods/168b83fa-b9ae-46e8-999b-bc60420d32c9/volumes"
Jul 16 23:00:39.814528 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:39.814457    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" path="/var/lib/kubelet/pods/d512aa0b-5a03-40c2-84c4-5ba1c55287eb/volumes"
Jul 16 23:00:41.875552 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:41.875522    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:46.946226 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:00:46.946191    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:00.335436 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335403    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"]
Jul 16 23:01:00.335890 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335786    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:01:00.335890 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335797    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:01:00.335890 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335815    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:01:00.335890 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335820    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:01:00.335890 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335838    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168b83fa-b9ae-46e8-999b-bc60420d32c9" containerName="maas-api"
Jul 16 23:01:00.335890 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335844    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="168b83fa-b9ae-46e8-999b-bc60420d32c9" containerName="maas-api"
Jul 16 23:01:00.336082 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335911    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:01:00.336082 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335920    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:01:00.336082 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335927    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="168b83fa-b9ae-46e8-999b-bc60420d32c9" containerName="maas-api"
Jul 16 23:01:00.336082 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.335933    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:01:00.339031 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.339015    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"
Jul 16 23:01:00.341752 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.341732    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tqg8r\""
Jul 16 23:01:00.349751 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.349727    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"]
Jul 16 23:01:00.369558 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.369532    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05af67f-0e62-466a-a60e-23661440802f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj\" (UID: \"e05af67f-0e62-466a-a60e-23661440802f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"
Jul 16 23:01:00.470707 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.470669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05af67f-0e62-466a-a60e-23661440802f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj\" (UID: \"e05af67f-0e62-466a-a60e-23661440802f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"
Jul 16 23:01:00.471358 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.471341    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05af67f-0e62-466a-a60e-23661440802f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj\" (UID: \"e05af67f-0e62-466a-a60e-23661440802f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"
Jul 16 23:01:00.650281 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.650205    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"
Jul 16 23:01:00.786133 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:00.786005    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"]
Jul 16 23:01:00.788593 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:01:00.788567    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode05af67f_0e62_466a_a60e_23661440802f.slice/crio-367b4f443b42dd8b13082a5d8123abfdd00324334e084e8b21652bb1bc1ea646 WatchSource:0}: Error finding container 367b4f443b42dd8b13082a5d8123abfdd00324334e084e8b21652bb1bc1ea646: Status 404 returned error can't find the container with id 367b4f443b42dd8b13082a5d8123abfdd00324334e084e8b21652bb1bc1ea646
Jul 16 23:01:01.276055 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:01.275970    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" event={"ID":"e05af67f-0e62-466a-a60e-23661440802f","Type":"ContainerStarted","Data":"310ff77df4325c5d9204c1e4fcf071e95c1c29c4513b3c3f2fb5ee7532c03097"}
Jul 16 23:01:01.276055 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:01.276021    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" event={"ID":"e05af67f-0e62-466a-a60e-23661440802f","Type":"ContainerStarted","Data":"367b4f443b42dd8b13082a5d8123abfdd00324334e084e8b21652bb1bc1ea646"}
Jul 16 23:01:01.296709 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:01.296662    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" podStartSLOduration=1.296647289 podStartE2EDuration="1.296647289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:01:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:01:01.294404111 +0000 UTC m=+2653.967663139" watchObservedRunningTime="2026-07-16 23:01:01.296647289 +0000 UTC m=+2653.969906297"
Jul 16 23:01:02.282132 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:02.282071    2556 generic.go:356] "Generic (PLEG): container finished" podID="e05af67f-0e62-466a-a60e-23661440802f" containerID="310ff77df4325c5d9204c1e4fcf071e95c1c29c4513b3c3f2fb5ee7532c03097" exitCode=6
Jul 16 23:01:02.282608 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:02.282181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" event={"ID":"e05af67f-0e62-466a-a60e-23661440802f","Type":"ContainerDied","Data":"310ff77df4325c5d9204c1e4fcf071e95c1c29c4513b3c3f2fb5ee7532c03097"}
Jul 16 23:01:02.282608 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:02.282359    2556 scope.go:117] "RemoveContainer" containerID="310ff77df4325c5d9204c1e4fcf071e95c1c29c4513b3c3f2fb5ee7532c03097"
Jul 16 23:01:03.288867 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:03.288830    2556 generic.go:356] "Generic (PLEG): container finished" podID="e05af67f-0e62-466a-a60e-23661440802f" containerID="5d4ce16f0a3cd51c9d1ffded8809de6773429580688c2d64362acb8fda833946" exitCode=6
Jul 16 23:01:03.289377 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:03.288876    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" event={"ID":"e05af67f-0e62-466a-a60e-23661440802f","Type":"ContainerDied","Data":"5d4ce16f0a3cd51c9d1ffded8809de6773429580688c2d64362acb8fda833946"}
Jul 16 23:01:03.289377 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:03.288937    2556 scope.go:117] "RemoveContainer" containerID="310ff77df4325c5d9204c1e4fcf071e95c1c29c4513b3c3f2fb5ee7532c03097"
Jul 16 23:01:03.289377 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:03.289285    2556 scope.go:117] "RemoveContainer" containerID="5d4ce16f0a3cd51c9d1ffded8809de6773429580688c2d64362acb8fda833946"
Jul 16 23:01:03.289553 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:01:03.289497    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj_odh-ai-gateway-infra(e05af67f-0e62-466a-a60e-23661440802f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" podUID="e05af67f-0e62-466a-a60e-23661440802f"
Jul 16 23:01:04.298699 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:04.298664    2556 scope.go:117] "RemoveContainer" containerID="5d4ce16f0a3cd51c9d1ffded8809de6773429580688c2d64362acb8fda833946"
Jul 16 23:01:04.299206 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:01:04.298895    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj_odh-ai-gateway-infra(e05af67f-0e62-466a-a60e-23661440802f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" podUID="e05af67f-0e62-466a-a60e-23661440802f"
Jul 16 23:01:12.706673 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:12.706624    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:12.808327 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:12.808289    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:15.807057 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:15.807031    2556 scope.go:117] "RemoveContainer" containerID="5d4ce16f0a3cd51c9d1ffded8809de6773429580688c2d64362acb8fda833946"
Jul 16 23:01:16.359345 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:16.359310    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" event={"ID":"e05af67f-0e62-466a-a60e-23661440802f","Type":"ContainerStarted","Data":"2f6093087254560463d9740fd6d86ac2fc7171b07b32f7686fb214ea81ab2c44"}
Jul 16 23:01:17.365633 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:17.365597    2556 generic.go:356] "Generic (PLEG): container finished" podID="e05af67f-0e62-466a-a60e-23661440802f" containerID="2f6093087254560463d9740fd6d86ac2fc7171b07b32f7686fb214ea81ab2c44" exitCode=6
Jul 16 23:01:17.366061 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:17.365667    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" event={"ID":"e05af67f-0e62-466a-a60e-23661440802f","Type":"ContainerDied","Data":"2f6093087254560463d9740fd6d86ac2fc7171b07b32f7686fb214ea81ab2c44"}
Jul 16 23:01:17.366061 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:17.365725    2556 scope.go:117] "RemoveContainer" containerID="5d4ce16f0a3cd51c9d1ffded8809de6773429580688c2d64362acb8fda833946"
Jul 16 23:01:17.366061 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:17.365990    2556 scope.go:117] "RemoveContainer" containerID="2f6093087254560463d9740fd6d86ac2fc7171b07b32f7686fb214ea81ab2c44"
Jul 16 23:01:17.366264 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:01:17.366235    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj_odh-ai-gateway-infra(e05af67f-0e62-466a-a60e-23661440802f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" podUID="e05af67f-0e62-466a-a60e-23661440802f"
Jul 16 23:01:17.385773 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:17.385752    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"]
Jul 16 23:01:18.510505 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:18.510481    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"
Jul 16 23:01:18.612852 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:18.612822    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05af67f-0e62-466a-a60e-23661440802f-maas-api-service-ca\") pod \"e05af67f-0e62-466a-a60e-23661440802f\" (UID: \"e05af67f-0e62-466a-a60e-23661440802f\") "
Jul 16 23:01:18.613167 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:18.613146    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05af67f-0e62-466a-a60e-23661440802f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e05af67f-0e62-466a-a60e-23661440802f" (UID: "e05af67f-0e62-466a-a60e-23661440802f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:01:18.713970 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:18.713908    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05af67f-0e62-466a-a60e-23661440802f-maas-api-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:01:19.378797 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:19.378767    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"
Jul 16 23:01:19.378797 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:19.378780    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj" event={"ID":"e05af67f-0e62-466a-a60e-23661440802f","Type":"ContainerDied","Data":"367b4f443b42dd8b13082a5d8123abfdd00324334e084e8b21652bb1bc1ea646"}
Jul 16 23:01:19.379024 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:19.378838    2556 scope.go:117] "RemoveContainer" containerID="2f6093087254560463d9740fd6d86ac2fc7171b07b32f7686fb214ea81ab2c44"
Jul 16 23:01:19.400205 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:19.400181    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"]
Jul 16 23:01:19.407200 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:19.407178    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-bf38b381-pgjsj"]
Jul 16 23:01:19.814389 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:19.814353    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e05af67f-0e62-466a-a60e-23661440802f" path="/var/lib/kubelet/pods/e05af67f-0e62-466a-a60e-23661440802f/volumes"
Jul 16 23:01:23.929244 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:23.929211    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:24.069435 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:24.069403    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:30.133232 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:30.133197    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:30.235537 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:30.235488    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:34.101028 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:34.100998    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:34.314632 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:34.314600    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:35.363274 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:35.363235    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:35.459986 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:35.459954    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:36.257780 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:36.257737    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:36.356290 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:36.356260    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:36.459333 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:36.459300    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:37.777773 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:37.777733    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:43.783701 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:43.783587    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:43.883471 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:43.883435    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:44.441427 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:44.441396    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:47.981178 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:47.981154    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 23:01:47.985505 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:47.985481    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 23:01:47.986681 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:47.986659    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 23:01:47.990747 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:47.990732    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 23:01:50.409173 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:50.409139    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:50.522867 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:50.522680    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:50.615478 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:01:50.615435    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:01.270403 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:01.270364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:01.495400 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:01.495361    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:09.494007 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:09.493965    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:22.431515 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:22.431479    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:24.225818 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:24.225783    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:28.329390 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:28.329355    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:28.393401 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:28.393367    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:28.662691 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:28.662593    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:28.955942 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:28.955831    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:29.176193 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:29.176154    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:31.442260 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.442224    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.529803    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"]
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530843    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530864    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530888    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530896    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d512aa0b-5a03-40c2-84c4-5ba1c55287eb" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530919    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530926    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530962    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.530975    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.531203    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.531224    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:31.538202 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.531970    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:31.539899 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.539838    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.543813 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.543061    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-bf84b0-serving-cert\""
Jul 16 23:02:31.543813 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.543671    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"]
Jul 16 23:02:31.545854 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.545650    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 23:02:31.595463 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.595427    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:31.660058 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.660019    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd23625-a5a5-4568-8817-10661b307562-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.660252 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.660079    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl54t\" (UniqueName: \"kubernetes.io/projected/fcd23625-a5a5-4568-8817-10661b307562-kube-api-access-kl54t\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.660252 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.660188    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.760765 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.760669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.760940 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.760777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd23625-a5a5-4568-8817-10661b307562-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.760940 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.760806    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl54t\" (UniqueName: \"kubernetes.io/projected/fcd23625-a5a5-4568-8817-10661b307562-kube-api-access-kl54t\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.760940 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:02:31.760826    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-serving-cert: secret "maas-api-e2e-inf-a-bf84b0-serving-cert" not found
Jul 16 23:02:31.760940 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:02:31.760901    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls podName:fcd23625-a5a5-4568-8817-10661b307562 nodeName:}" failed. No retries permitted until 2026-07-16 23:02:32.260880887 +0000 UTC m=+2744.934139894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls") pod "maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht" (UID: "fcd23625-a5a5-4568-8817-10661b307562") : secret "maas-api-e2e-inf-a-bf84b0-serving-cert" not found
Jul 16 23:02:31.761480 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.761462    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd23625-a5a5-4568-8817-10661b307562-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.771873 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.771851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl54t\" (UniqueName: \"kubernetes.io/projected/fcd23625-a5a5-4568-8817-10661b307562-kube-api-access-kl54t\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:31.918933 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:31.918902    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:32.117581 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:32.117549    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:32.219136 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:32.219074    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:32.265401 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:32.265370    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:32.267980 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:32.267960    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls\") pod \"maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:32.454724 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:32.454635    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:32.592553 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:32.592529    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"]
Jul 16 23:02:32.595080 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:02:32.595055    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd23625_a5a5_4568_8817_10661b307562.slice/crio-b3d63b52bdf2045d4515043a9395de18bb10607c3e9cefdace80765db17dec6a WatchSource:0}: Error finding container b3d63b52bdf2045d4515043a9395de18bb10607c3e9cefdace80765db17dec6a: Status 404 returned error can't find the container with id b3d63b52bdf2045d4515043a9395de18bb10607c3e9cefdace80765db17dec6a
Jul 16 23:02:32.749147 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:32.749025    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht" event={"ID":"fcd23625-a5a5-4568-8817-10661b307562","Type":"ContainerStarted","Data":"b3d63b52bdf2045d4515043a9395de18bb10607c3e9cefdace80765db17dec6a"}
Jul 16 23:02:33.755603 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:33.755519    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht" event={"ID":"fcd23625-a5a5-4568-8817-10661b307562","Type":"ContainerStarted","Data":"c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8"}
Jul 16 23:02:33.755946 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:33.755638    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:33.774447 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:33.774398    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht" podStartSLOduration=1.915978089 podStartE2EDuration="2.774382232s" totalImagesPullingTime="858.404143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:02:33.774085208 +0000 UTC m=+2746.447344245" watchObservedRunningTime="2026-07-16 23:02:33.774382232 +0000 UTC m=+2746.447641238"
Jul 16 23:02:35.926510 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:35.926473    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:36.255067 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:36.254978    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:36.546383 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:36.546344    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:37.248397 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:37.248368    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:39.595898 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.595864    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"]
Jul 16 23:02:39.596428 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.596347    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05af67f-0e62-466a-a60e-23661440802f" containerName="revoke-keys"
Jul 16 23:02:39.599716 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.599689    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.601933 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.601908    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-30cb29-serving-cert\""
Jul 16 23:02:39.610639 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.610613    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"]
Jul 16 23:02:39.628002 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.627973    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:39.693959 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.693926    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:39.736217 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.736173    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.736407 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.736225    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpskx\" (UniqueName: \"kubernetes.io/projected/c5a8a684-297e-40e0-953b-40451b7b4314-kube-api-access-bpskx\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.736407 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.736276    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a8a684-297e-40e0-953b-40451b7b4314-openshift-service-ca\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.767006 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.766975    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:02:39.837010 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.836978    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.837223 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.837019    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpskx\" (UniqueName: \"kubernetes.io/projected/c5a8a684-297e-40e0-953b-40451b7b4314-kube-api-access-bpskx\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.837223 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.837060    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a8a684-297e-40e0-953b-40451b7b4314-openshift-service-ca\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.837223 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:02:39.837153    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-serving-cert: secret "maas-api-e2e-inf-b-30cb29-serving-cert" not found
Jul 16 23:02:39.837404 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:02:39.837235    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls podName:c5a8a684-297e-40e0-953b-40451b7b4314 nodeName:}" failed. No retries permitted until 2026-07-16 23:02:40.337214973 +0000 UTC m=+2753.010473959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls") pod "maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq" (UID: "c5a8a684-297e-40e0-953b-40451b7b4314") : secret "maas-api-e2e-inf-b-30cb29-serving-cert" not found
Jul 16 23:02:39.837831 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.837802    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a8a684-297e-40e0-953b-40451b7b4314-openshift-service-ca\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:39.845856 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:39.845831    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpskx\" (UniqueName: \"kubernetes.io/projected/c5a8a684-297e-40e0-953b-40451b7b4314-kube-api-access-bpskx\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:40.001833 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.001744    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:40.331297 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.331257    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:40.341253 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.341223    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:40.343812 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.343789    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls\") pod \"maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:40.440377 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.440344    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:40.511570 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.511535    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:40.658058 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.658013    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"]
Jul 16 23:02:40.660627 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:02:40.660598    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5a8a684_297e_40e0_953b_40451b7b4314.slice/crio-8c2848b041b39fb8bc375b4a70576522b374eb4f254c9e24281a5c742673de47 WatchSource:0}: Error finding container 8c2848b041b39fb8bc375b4a70576522b374eb4f254c9e24281a5c742673de47: Status 404 returned error can't find the container with id 8c2848b041b39fb8bc375b4a70576522b374eb4f254c9e24281a5c742673de47
Jul 16 23:02:40.799925 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:40.799890    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq" event={"ID":"c5a8a684-297e-40e0-953b-40451b7b4314","Type":"ContainerStarted","Data":"8c2848b041b39fb8bc375b4a70576522b374eb4f254c9e24281a5c742673de47"}
Jul 16 23:02:41.811367 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:41.811327    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq" event={"ID":"c5a8a684-297e-40e0-953b-40451b7b4314","Type":"ContainerStarted","Data":"0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f"}
Jul 16 23:02:41.811744 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:41.811390    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:41.824678 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:41.824624    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq" podStartSLOduration=2.412996757 podStartE2EDuration="2.82460519s" totalImagesPullingTime="411.608433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:02:41.823139097 +0000 UTC m=+2754.496398130" watchObservedRunningTime="2026-07-16 23:02:41.82460519 +0000 UTC m=+2754.497864197"
Jul 16 23:02:43.995307 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:43.995272    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:47.818427 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:47.818400    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:02:52.986268 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:52.986233    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:53.089574 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:02:53.089527    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:02.357089 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:02.357053    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:02.462669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:02.462636    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:15.213150 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:15.213097    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:42.780135 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.780069    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"]
Jul 16 23:03:42.784246 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.784221    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.786985 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.786966    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-30cb29\"/\"kube-root-ca.crt\""
Jul 16 23:03:42.787696 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.787661    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-30cb29\"/\"default-dockercfg-7742d\""
Jul 16 23:03:42.787812 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.787754    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-30cb29\"/\"test-model-30cb29-kserve-self-signed-certs\""
Jul 16 23:03:42.787812 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.787780    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-30cb29\"/\"openshift-service-ca.crt\""
Jul 16 23:03:42.797766 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.797737    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"]
Jul 16 23:03:42.804330 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.804304    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e51857f-d37b-421e-a21a-c9ec3676fba7-tls-certs\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.804476 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.804334    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-home\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.804476 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.804360    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-dshm\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.804476 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.804456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-model-cache\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.804651 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.804525    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjvgp\" (UniqueName: \"kubernetes.io/projected/3e51857f-d37b-421e-a21a-c9ec3676fba7-kube-api-access-tjvgp\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.804651 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.804558    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-tmp-dir\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.818311 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.818275    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:42.905608 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.905563    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-dshm\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.905790 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.905626    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-model-cache\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.905790 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.905660    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjvgp\" (UniqueName: \"kubernetes.io/projected/3e51857f-d37b-421e-a21a-c9ec3676fba7-kube-api-access-tjvgp\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.905790 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.905679    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-tmp-dir\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.905790 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.905755    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e51857f-d37b-421e-a21a-c9ec3676fba7-tls-certs\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.905790 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.905780    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-home\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.906165 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.906102    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-model-cache\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.906289 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.906142    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-tmp-dir\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.906289 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.906199    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-home\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.908030 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.908011    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-dshm\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.908487 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.908469    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e51857f-d37b-421e-a21a-c9ec3676fba7-tls-certs\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:42.914692 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:42.914666    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjvgp\" (UniqueName: \"kubernetes.io/projected/3e51857f-d37b-421e-a21a-c9ec3676fba7-kube-api-access-tjvgp\") pod \"test-model-30cb29-kserve-559dc46487-r2866\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") " pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:43.094669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:43.094631    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:43.240644 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:43.240618    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"]
Jul 16 23:03:43.242040 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:03:43.242009    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e51857f_d37b_421e_a21a_c9ec3676fba7.slice/crio-4f0148b563c43620fc490b0db0f700b4a0f491fec590c924a2cbd63473552909 WatchSource:0}: Error finding container 4f0148b563c43620fc490b0db0f700b4a0f491fec590c924a2cbd63473552909: Status 404 returned error can't find the container with id 4f0148b563c43620fc490b0db0f700b4a0f491fec590c924a2cbd63473552909
Jul 16 23:03:44.129431 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:44.129388    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866" event={"ID":"3e51857f-d37b-421e-a21a-c9ec3676fba7","Type":"ContainerStarted","Data":"7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22"}
Jul 16 23:03:44.129852 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:44.129446    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866" event={"ID":"3e51857f-d37b-421e-a21a-c9ec3676fba7","Type":"ContainerStarted","Data":"4f0148b563c43620fc490b0db0f700b4a0f491fec590c924a2cbd63473552909"}
Jul 16 23:03:44.165099 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:44.165026    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866" podStartSLOduration=2.165010053 podStartE2EDuration="2.165010053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:03:44.164915912 +0000 UTC m=+2816.838174929" watchObservedRunningTime="2026-07-16 23:03:44.165010053 +0000 UTC m=+2816.838269062"
Jul 16 23:03:53.095408 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:53.095357    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:53.095798 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:53.095423    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:53.107953 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:53.107927    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:03:53.185750 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:03:53.185718    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:04:03.771134 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:03.771088    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:04.660735 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:04.660698    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"]
Jul 16 23:04:04.683026 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:04.682997    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:05.323380 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:05.323333    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:05.421533 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:05.421496    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:15.304638 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.304596    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"]
Jul 16 23:04:15.305230 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.304886    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht" podUID="fcd23625-a5a5-4568-8817-10661b307562" containerName="maas-api" containerID="cri-o://c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8" gracePeriod=30
Jul 16 23:04:15.332127 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.332067    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:15.573566 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.573534    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:04:15.724802 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.724710    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl54t\" (UniqueName: \"kubernetes.io/projected/fcd23625-a5a5-4568-8817-10661b307562-kube-api-access-kl54t\") pod \"fcd23625-a5a5-4568-8817-10661b307562\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") "
Jul 16 23:04:15.724802 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.724778    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls\") pod \"fcd23625-a5a5-4568-8817-10661b307562\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") "
Jul 16 23:04:15.725052 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.724812    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd23625-a5a5-4568-8817-10661b307562-openshift-service-ca\") pod \"fcd23625-a5a5-4568-8817-10661b307562\" (UID: \"fcd23625-a5a5-4568-8817-10661b307562\") "
Jul 16 23:04:15.725326 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.725299    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd23625-a5a5-4568-8817-10661b307562-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fcd23625-a5a5-4568-8817-10661b307562" (UID: "fcd23625-a5a5-4568-8817-10661b307562"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:04:15.727200 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.727172    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fcd23625-a5a5-4568-8817-10661b307562" (UID: "fcd23625-a5a5-4568-8817-10661b307562"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:04:15.727328 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.727283    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd23625-a5a5-4568-8817-10661b307562-kube-api-access-kl54t" (OuterVolumeSpecName: "kube-api-access-kl54t") pod "fcd23625-a5a5-4568-8817-10661b307562" (UID: "fcd23625-a5a5-4568-8817-10661b307562"). InnerVolumeSpecName "kube-api-access-kl54t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:04:15.826444 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.826407    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl54t\" (UniqueName: \"kubernetes.io/projected/fcd23625-a5a5-4568-8817-10661b307562-kube-api-access-kl54t\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:15.826444 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.826435    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcd23625-a5a5-4568-8817-10661b307562-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:15.826444 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:15.826445    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcd23625-a5a5-4568-8817-10661b307562-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:16.293398 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.293365    2556 generic.go:356] "Generic (PLEG): container finished" podID="fcd23625-a5a5-4568-8817-10661b307562" containerID="c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8" exitCode=0
Jul 16 23:04:16.293623 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.293417    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht" event={"ID":"fcd23625-a5a5-4568-8817-10661b307562","Type":"ContainerDied","Data":"c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8"}
Jul 16 23:04:16.293623 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.293423    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"
Jul 16 23:04:16.293623 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.293482    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht" event={"ID":"fcd23625-a5a5-4568-8817-10661b307562","Type":"ContainerDied","Data":"b3d63b52bdf2045d4515043a9395de18bb10607c3e9cefdace80765db17dec6a"}
Jul 16 23:04:16.293623 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.293505    2556 scope.go:117] "RemoveContainer" containerID="c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8"
Jul 16 23:04:16.302643 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.302626    2556 scope.go:117] "RemoveContainer" containerID="c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8"
Jul 16 23:04:16.302887 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:04:16.302865    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8\": container with ID starting with c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8 not found: ID does not exist" containerID="c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8"
Jul 16 23:04:16.302939 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.302896    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8"} err="failed to get container status \"c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8\": rpc error: code = NotFound desc = could not find container \"c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8\": container with ID starting with c671a1fb0c03978439557e01a786c5378c1e2e7de067916646a49c0c618eb8a8 not found: ID does not exist"
Jul 16 23:04:16.309408 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.309384    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"]
Jul 16 23:04:16.313520 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:16.313497    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bf84b0-96cb88b5f-jdcht"]
Jul 16 23:04:17.811704 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:17.811673    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcd23625-a5a5-4568-8817-10661b307562" path="/var/lib/kubelet/pods/fcd23625-a5a5-4568-8817-10661b307562/volumes"
Jul 16 23:04:19.682092 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:19.682046    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866" podUID="3e51857f-d37b-421e-a21a-c9ec3676fba7" containerName="main" containerID="cri-o://7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22" gracePeriod=45
Jul 16 23:04:19.882357 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:19.882313    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:19.951520 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:19.951496    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:04:20.062982 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.062948    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjvgp\" (UniqueName: \"kubernetes.io/projected/3e51857f-d37b-421e-a21a-c9ec3676fba7-kube-api-access-tjvgp\") pod \"3e51857f-d37b-421e-a21a-c9ec3676fba7\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") "
Jul 16 23:04:20.063197 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063031    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-home\") pod \"3e51857f-d37b-421e-a21a-c9ec3676fba7\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") "
Jul 16 23:04:20.063197 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063048    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-tmp-dir\") pod \"3e51857f-d37b-421e-a21a-c9ec3676fba7\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") "
Jul 16 23:04:20.063197 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063096    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e51857f-d37b-421e-a21a-c9ec3676fba7-tls-certs\") pod \"3e51857f-d37b-421e-a21a-c9ec3676fba7\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") "
Jul 16 23:04:20.063197 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063156    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-model-cache\") pod \"3e51857f-d37b-421e-a21a-c9ec3676fba7\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") "
Jul 16 23:04:20.063346 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063205    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-dshm\") pod \"3e51857f-d37b-421e-a21a-c9ec3676fba7\" (UID: \"3e51857f-d37b-421e-a21a-c9ec3676fba7\") "
Jul 16 23:04:20.063386 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063351    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-home" (OuterVolumeSpecName: "home") pod "3e51857f-d37b-421e-a21a-c9ec3676fba7" (UID: "3e51857f-d37b-421e-a21a-c9ec3676fba7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:20.063386 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063367    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e51857f-d37b-421e-a21a-c9ec3676fba7" (UID: "3e51857f-d37b-421e-a21a-c9ec3676fba7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:20.063598 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063576    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-home\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:20.063661 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063604    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-tmp-dir\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:20.063661 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.063584    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e51857f-d37b-421e-a21a-c9ec3676fba7" (UID: "3e51857f-d37b-421e-a21a-c9ec3676fba7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:20.065516 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.065489    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-dshm" (OuterVolumeSpecName: "dshm") pod "3e51857f-d37b-421e-a21a-c9ec3676fba7" (UID: "3e51857f-d37b-421e-a21a-c9ec3676fba7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:04:20.065623 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.065532    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e51857f-d37b-421e-a21a-c9ec3676fba7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e51857f-d37b-421e-a21a-c9ec3676fba7" (UID: "3e51857f-d37b-421e-a21a-c9ec3676fba7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:04:20.065623 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.065546    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e51857f-d37b-421e-a21a-c9ec3676fba7-kube-api-access-tjvgp" (OuterVolumeSpecName: "kube-api-access-tjvgp") pod "3e51857f-d37b-421e-a21a-c9ec3676fba7" (UID: "3e51857f-d37b-421e-a21a-c9ec3676fba7"). InnerVolumeSpecName "kube-api-access-tjvgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:04:20.164398 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.164354    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e51857f-d37b-421e-a21a-c9ec3676fba7-tls-certs\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:20.164398 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.164392    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-model-cache\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:20.164398 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.164401    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e51857f-d37b-421e-a21a-c9ec3676fba7-dshm\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:20.164669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.164409    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjvgp\" (UniqueName: \"kubernetes.io/projected/3e51857f-d37b-421e-a21a-c9ec3676fba7-kube-api-access-tjvgp\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:20.316232 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.316193    2556 generic.go:356] "Generic (PLEG): container finished" podID="3e51857f-d37b-421e-a21a-c9ec3676fba7" containerID="7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22" exitCode=0
Jul 16 23:04:20.316445 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.316248    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"
Jul 16 23:04:20.316445 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.316287    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866" event={"ID":"3e51857f-d37b-421e-a21a-c9ec3676fba7","Type":"ContainerDied","Data":"7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22"}
Jul 16 23:04:20.316445 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.316335    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866" event={"ID":"3e51857f-d37b-421e-a21a-c9ec3676fba7","Type":"ContainerDied","Data":"4f0148b563c43620fc490b0db0f700b4a0f491fec590c924a2cbd63473552909"}
Jul 16 23:04:20.316445 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.316359    2556 scope.go:117] "RemoveContainer" containerID="7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22"
Jul 16 23:04:20.327271 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.327249    2556 scope.go:117] "RemoveContainer" containerID="7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22"
Jul 16 23:04:20.327620 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:04:20.327599    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22\": container with ID starting with 7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22 not found: ID does not exist" containerID="7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22"
Jul 16 23:04:20.327679 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.327627    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22"} err="failed to get container status \"7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22\": rpc error: code = NotFound desc = could not find container \"7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22\": container with ID starting with 7066a1c36d40adf1867a8a2db6060fa53269593d1337fa169769887185abfa22 not found: ID does not exist"
Jul 16 23:04:20.338245 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.338175    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"]
Jul 16 23:04:20.340494 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:20.340470    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-30cb29/test-model-30cb29-kserve-559dc46487-r2866"]
Jul 16 23:04:21.812839 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:21.812805    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e51857f-d37b-421e-a21a-c9ec3676fba7" path="/var/lib/kubelet/pods/3e51857f-d37b-421e-a21a-c9ec3676fba7/volumes"
Jul 16 23:04:25.642560 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:25.642525    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:25.715456 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:25.715416    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:30.591371 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:30.591335    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:30.802686 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:30.802651    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:31.425711 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:31.425674    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:31.526857 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:31.526809    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:41.320852 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.320745    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"]
Jul 16 23:04:41.321313 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.321015    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq" podUID="c5a8a684-297e-40e0-953b-40451b7b4314" containerName="maas-api" containerID="cri-o://0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f" gracePeriod=30
Jul 16 23:04:41.351151 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.351095    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:41.782963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.782925    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:04:41.867896 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.867795    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls\") pod \"c5a8a684-297e-40e0-953b-40451b7b4314\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") "
Jul 16 23:04:41.867896 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.867891    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a8a684-297e-40e0-953b-40451b7b4314-openshift-service-ca\") pod \"c5a8a684-297e-40e0-953b-40451b7b4314\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") "
Jul 16 23:04:41.868171 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.867974    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpskx\" (UniqueName: \"kubernetes.io/projected/c5a8a684-297e-40e0-953b-40451b7b4314-kube-api-access-bpskx\") pod \"c5a8a684-297e-40e0-953b-40451b7b4314\" (UID: \"c5a8a684-297e-40e0-953b-40451b7b4314\") "
Jul 16 23:04:41.868313 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.868285    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5a8a684-297e-40e0-953b-40451b7b4314-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5a8a684-297e-40e0-953b-40451b7b4314" (UID: "c5a8a684-297e-40e0-953b-40451b7b4314"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:04:41.870186 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.870160    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5a8a684-297e-40e0-953b-40451b7b4314" (UID: "c5a8a684-297e-40e0-953b-40451b7b4314"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:04:41.870326 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.870299    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5a8a684-297e-40e0-953b-40451b7b4314-kube-api-access-bpskx" (OuterVolumeSpecName: "kube-api-access-bpskx") pod "c5a8a684-297e-40e0-953b-40451b7b4314" (UID: "c5a8a684-297e-40e0-953b-40451b7b4314"). InnerVolumeSpecName "kube-api-access-bpskx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:04:41.969486 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.969440    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpskx\" (UniqueName: \"kubernetes.io/projected/c5a8a684-297e-40e0-953b-40451b7b4314-kube-api-access-bpskx\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:41.969486 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.969478    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a8a684-297e-40e0-953b-40451b7b4314-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:41.969486 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:41.969488    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a8a684-297e-40e0-953b-40451b7b4314-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:04:42.441847 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.441803    2556 generic.go:356] "Generic (PLEG): container finished" podID="c5a8a684-297e-40e0-953b-40451b7b4314" containerID="0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f" exitCode=0
Jul 16 23:04:42.442352 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.441862    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"
Jul 16 23:04:42.442352 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.441873    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq" event={"ID":"c5a8a684-297e-40e0-953b-40451b7b4314","Type":"ContainerDied","Data":"0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f"}
Jul 16 23:04:42.442352 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.441930    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq" event={"ID":"c5a8a684-297e-40e0-953b-40451b7b4314","Type":"ContainerDied","Data":"8c2848b041b39fb8bc375b4a70576522b374eb4f254c9e24281a5c742673de47"}
Jul 16 23:04:42.442352 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.441957    2556 scope.go:117] "RemoveContainer" containerID="0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f"
Jul 16 23:04:42.452572 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.452552    2556 scope.go:117] "RemoveContainer" containerID="0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f"
Jul 16 23:04:42.452900 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:04:42.452878    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f\": container with ID starting with 0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f not found: ID does not exist" containerID="0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f"
Jul 16 23:04:42.452944 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.452910    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f"} err="failed to get container status \"0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f\": rpc error: code = NotFound desc = could not find container \"0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f\": container with ID starting with 0dbbd9c96e0b891b823d0cd5d50810e9464fe871ea74740cdc672258bbf9b16f not found: ID does not exist"
Jul 16 23:04:42.463980 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.463946    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"]
Jul 16 23:04:42.467938 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:42.467907    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30cb29-79b9d4fc49-dx4hq"]
Jul 16 23:04:43.814583 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:43.814549    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5a8a684-297e-40e0-953b-40451b7b4314" path="/var/lib/kubelet/pods/c5a8a684-297e-40e0-953b-40451b7b4314/volumes"
Jul 16 23:04:45.999004 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:45.998950    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:51.779157 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:51.779101    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:51.842161 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:51.842088    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:56.676773 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:56.676742    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:56.907729 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:56.907684    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:57.224152 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:57.224106    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:57.532354 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:57.532264    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:04:57.970610 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:04:57.970573    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:00.442595 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.442557    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:00.569759 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.569726    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"]
Jul 16 23:05:00.570168 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570153    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd23625-a5a5-4568-8817-10661b307562" containerName="maas-api"
Jul 16 23:05:00.570230 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570171    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd23625-a5a5-4568-8817-10661b307562" containerName="maas-api"
Jul 16 23:05:00.570230 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570194    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e51857f-d37b-421e-a21a-c9ec3676fba7" containerName="main"
Jul 16 23:05:00.570230 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570200    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e51857f-d37b-421e-a21a-c9ec3676fba7" containerName="main"
Jul 16 23:05:00.570230 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570212    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5a8a684-297e-40e0-953b-40451b7b4314" containerName="maas-api"
Jul 16 23:05:00.570230 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570218    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5a8a684-297e-40e0-953b-40451b7b4314" containerName="maas-api"
Jul 16 23:05:00.570404 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570278    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e51857f-d37b-421e-a21a-c9ec3676fba7" containerName="main"
Jul 16 23:05:00.570404 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570285    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5a8a684-297e-40e0-953b-40451b7b4314" containerName="maas-api"
Jul 16 23:05:00.570404 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.570295    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcd23625-a5a5-4568-8817-10661b307562" containerName="maas-api"
Jul 16 23:05:00.573465 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.573447    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.578277 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.578254    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b9c909-serving-cert\""
Jul 16 23:05:00.579409 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.579388    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 23:05:00.589690 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.589662    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"]
Jul 16 23:05:00.632025 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.631987    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:00.748069 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.747970    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487a32a1-a2e9-4754-a0e8-82818850f895-maas-api-tls\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.748069 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.748027    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487a32a1-a2e9-4754-a0e8-82818850f895-openshift-service-ca\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.748325 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.748166    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtn8m\" (UniqueName: \"kubernetes.io/projected/487a32a1-a2e9-4754-a0e8-82818850f895-kube-api-access-rtn8m\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.748937 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.748909    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:00.849496 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.849458    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487a32a1-a2e9-4754-a0e8-82818850f895-maas-api-tls\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.849496 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.849502    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487a32a1-a2e9-4754-a0e8-82818850f895-openshift-service-ca\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.849780 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.849563    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtn8m\" (UniqueName: \"kubernetes.io/projected/487a32a1-a2e9-4754-a0e8-82818850f895-kube-api-access-rtn8m\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.850316 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.850292    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487a32a1-a2e9-4754-a0e8-82818850f895-openshift-service-ca\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.852242 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.852220    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487a32a1-a2e9-4754-a0e8-82818850f895-maas-api-tls\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.855030 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.855003    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:00.858192 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.858165    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtn8m\" (UniqueName: \"kubernetes.io/projected/487a32a1-a2e9-4754-a0e8-82818850f895-kube-api-access-rtn8m\") pod \"maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:00.884714 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:00.884683    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:01.049979 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:01.049945    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"]
Jul 16 23:05:01.052425 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:05:01.052374    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487a32a1_a2e9_4754_a0e8_82818850f895.slice/crio-139fb5f82784dd4eb017d4572fee52e5061df963e7343e275197f5768cee4ecd WatchSource:0}: Error finding container 139fb5f82784dd4eb017d4572fee52e5061df963e7343e275197f5768cee4ecd: Status 404 returned error can't find the container with id 139fb5f82784dd4eb017d4572fee52e5061df963e7343e275197f5768cee4ecd
Jul 16 23:05:01.053938 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:01.053913    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 23:05:01.384793 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:01.384696    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:01.486911 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:01.486860    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:01.553920 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:01.553888    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r" event={"ID":"487a32a1-a2e9-4754-a0e8-82818850f895","Type":"ContainerStarted","Data":"139fb5f82784dd4eb017d4572fee52e5061df963e7343e275197f5768cee4ecd"}
Jul 16 23:05:02.562406 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:02.562367    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r" event={"ID":"487a32a1-a2e9-4754-a0e8-82818850f895","Type":"ContainerStarted","Data":"2eb820ee9f9f4eff5ebe376bde02bf8551f82dbe87f37ed4b4dd7238f9f756a4"}
Jul 16 23:05:02.562865 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:02.562448    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:02.593879 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:02.593818    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r" podStartSLOduration=1.8520170390000001 podStartE2EDuration="2.593799109s" totalImagesPullingTime="741.78207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:05:02.592896556 +0000 UTC m=+2895.266155564" watchObservedRunningTime="2026-07-16 23:05:02.593799109 +0000 UTC m=+2895.267058118"
Jul 16 23:05:04.897423 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:04.897384    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:05.542542 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:05.542505    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:05.664332 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:05.664297    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:05.762411 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:05.762374    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:08.421898 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.421817    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:08.513604 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.513569    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"]
Jul 16 23:05:08.517514 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.517490    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.518634 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.518603    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-openshift-service-ca\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.518737 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.518693    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66cnz\" (UniqueName: \"kubernetes.io/projected/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-kube-api-access-66cnz\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.518806 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.518784    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.519879 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.519835    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8c14db-serving-cert\""
Jul 16 23:05:08.529807 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.529774    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"]
Jul 16 23:05:08.573880 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.573849    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:08.579484 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.579424    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:08.619572 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.619533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-openshift-service-ca\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.619785 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.619615    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66cnz\" (UniqueName: \"kubernetes.io/projected/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-kube-api-access-66cnz\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.619785 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.619718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.619922 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:05:08.619843    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-serving-cert: secret "maas-api-e2e-api-b-8c14db-serving-cert" not found
Jul 16 23:05:08.619922 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:05:08.619907    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls podName:8cd78f60-e7f4-4fdd-90a1-06666cc3dba2 nodeName:}" failed. No retries permitted until 2026-07-16 23:05:09.119884827 +0000 UTC m=+2901.793143828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls") pod "maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj" (UID: "8cd78f60-e7f4-4fdd-90a1-06666cc3dba2") : secret "maas-api-e2e-api-b-8c14db-serving-cert" not found
Jul 16 23:05:08.620538 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.620499    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-openshift-service-ca\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.630896 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.630865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66cnz\" (UniqueName: \"kubernetes.io/projected/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-kube-api-access-66cnz\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:08.679675 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:08.679581    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:09.022102 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.022010    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:09.124322 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.124281    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:09.127228 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.127195    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls\") pod \"maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:09.131998 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.131974    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:09.226765 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.226699    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:09.287801 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.287769    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"]
Jul 16 23:05:09.290553 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:05:09.290527    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd78f60_e7f4_4fdd_90a1_06666cc3dba2.slice/crio-ecfdf1b88a4861623c21a13ea137cabc8a6c0a5bcaf4834f44319f011ac92e9f WatchSource:0}: Error finding container ecfdf1b88a4861623c21a13ea137cabc8a6c0a5bcaf4834f44319f011ac92e9f: Status 404 returned error can't find the container with id ecfdf1b88a4861623c21a13ea137cabc8a6c0a5bcaf4834f44319f011ac92e9f
Jul 16 23:05:09.302765 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.302731    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:09.599221 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:09.599145    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj" event={"ID":"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2","Type":"ContainerStarted","Data":"ecfdf1b88a4861623c21a13ea137cabc8a6c0a5bcaf4834f44319f011ac92e9f"}
Jul 16 23:05:10.605975 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:10.605935    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj" event={"ID":"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2","Type":"ContainerStarted","Data":"7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472"}
Jul 16 23:05:10.606403 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:10.606005    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:10.625077 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:10.625008    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj" podStartSLOduration=2.304361881 podStartE2EDuration="2.624987211s" totalImagesPullingTime="320.62533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:05:10.622575986 +0000 UTC m=+2903.295834994" watchObservedRunningTime="2026-07-16 23:05:10.624987211 +0000 UTC m=+2903.298246220"
Jul 16 23:05:12.897713 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:12.897673    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:16.617048 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:16.617019    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:21.494850 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:21.494816    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:21.586748 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:21.586709    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:30.129440 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:30.129398    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:30.230886 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:30.230852    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:31.334017 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.333980    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:31.440756 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.440720    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:31.586740 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.586492    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"]
Jul 16 23:05:31.586912 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.586790    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r" podUID="487a32a1-a2e9-4754-a0e8-82818850f895" containerName="maas-api" containerID="cri-o://2eb820ee9f9f4eff5ebe376bde02bf8551f82dbe87f37ed4b4dd7238f9f756a4" gracePeriod=30
Jul 16 23:05:31.625963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.625931    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:31.729584 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.729554    2556 generic.go:356] "Generic (PLEG): container finished" podID="487a32a1-a2e9-4754-a0e8-82818850f895" containerID="2eb820ee9f9f4eff5ebe376bde02bf8551f82dbe87f37ed4b4dd7238f9f756a4" exitCode=0
Jul 16 23:05:31.729744 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.729608    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r" event={"ID":"487a32a1-a2e9-4754-a0e8-82818850f895","Type":"ContainerDied","Data":"2eb820ee9f9f4eff5ebe376bde02bf8551f82dbe87f37ed4b4dd7238f9f756a4"}
Jul 16 23:05:31.848078 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.848006    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:31.937316 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.937277    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487a32a1-a2e9-4754-a0e8-82818850f895-maas-api-tls\") pod \"487a32a1-a2e9-4754-a0e8-82818850f895\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") "
Jul 16 23:05:31.937492 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.937421    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487a32a1-a2e9-4754-a0e8-82818850f895-openshift-service-ca\") pod \"487a32a1-a2e9-4754-a0e8-82818850f895\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") "
Jul 16 23:05:31.937492 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.937450    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtn8m\" (UniqueName: \"kubernetes.io/projected/487a32a1-a2e9-4754-a0e8-82818850f895-kube-api-access-rtn8m\") pod \"487a32a1-a2e9-4754-a0e8-82818850f895\" (UID: \"487a32a1-a2e9-4754-a0e8-82818850f895\") "
Jul 16 23:05:31.937801 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.937772    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/487a32a1-a2e9-4754-a0e8-82818850f895-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "487a32a1-a2e9-4754-a0e8-82818850f895" (UID: "487a32a1-a2e9-4754-a0e8-82818850f895"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:05:31.939666 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.939636    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/487a32a1-a2e9-4754-a0e8-82818850f895-kube-api-access-rtn8m" (OuterVolumeSpecName: "kube-api-access-rtn8m") pod "487a32a1-a2e9-4754-a0e8-82818850f895" (UID: "487a32a1-a2e9-4754-a0e8-82818850f895"). InnerVolumeSpecName "kube-api-access-rtn8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:05:31.939756 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:31.939642    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/487a32a1-a2e9-4754-a0e8-82818850f895-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "487a32a1-a2e9-4754-a0e8-82818850f895" (UID: "487a32a1-a2e9-4754-a0e8-82818850f895"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:05:32.038547 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.038512    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/487a32a1-a2e9-4754-a0e8-82818850f895-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:05:32.038547 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.038544    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/487a32a1-a2e9-4754-a0e8-82818850f895-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:05:32.038547 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.038554    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtn8m\" (UniqueName: \"kubernetes.io/projected/487a32a1-a2e9-4754-a0e8-82818850f895-kube-api-access-rtn8m\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:05:32.739470 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.739439    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"
Jul 16 23:05:32.739470 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.739454    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r" event={"ID":"487a32a1-a2e9-4754-a0e8-82818850f895","Type":"ContainerDied","Data":"139fb5f82784dd4eb017d4572fee52e5061df963e7343e275197f5768cee4ecd"}
Jul 16 23:05:32.740046 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.739513    2556 scope.go:117] "RemoveContainer" containerID="2eb820ee9f9f4eff5ebe376bde02bf8551f82dbe87f37ed4b4dd7238f9f756a4"
Jul 16 23:05:32.760529 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.760470    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"]
Jul 16 23:05:32.762836 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:32.762812    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b9c909-7785c4c65d-28k4r"]
Jul 16 23:05:33.201210 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:33.201176    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:33.811845 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:33.811802    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="487a32a1-a2e9-4754-a0e8-82818850f895" path="/var/lib/kubelet/pods/487a32a1-a2e9-4754-a0e8-82818850f895/volumes"
Jul 16 23:05:35.883621 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:35.883575    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:41.958447 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:41.958411    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:42.058354 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:42.058317    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:46.653717 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:46.653681    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:47.796936 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:47.796879    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:48.145429 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:48.145339    2556 scope.go:117] "RemoveContainer" containerID="9e3399300c786a0bd89573390f72ead6a090f69da5679ab8ff791241196cc0cd"
Jul 16 23:05:57.545543 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.545493    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"]
Jul 16 23:05:57.546180 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.545768    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj" podUID="8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" containerName="maas-api" containerID="cri-o://7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472" gracePeriod=30
Jul 16 23:05:57.588740 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.588702    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:05:57.814415 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.814388    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:57.869353 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.869317    2556 generic.go:356] "Generic (PLEG): container finished" podID="8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" containerID="7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472" exitCode=0
Jul 16 23:05:57.869523 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.869366    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"
Jul 16 23:05:57.869523 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.869407    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj" event={"ID":"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2","Type":"ContainerDied","Data":"7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472"}
Jul 16 23:05:57.869523 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.869467    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj" event={"ID":"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2","Type":"ContainerDied","Data":"ecfdf1b88a4861623c21a13ea137cabc8a6c0a5bcaf4834f44319f011ac92e9f"}
Jul 16 23:05:57.869523 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.869489    2556 scope.go:117] "RemoveContainer" containerID="7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472"
Jul 16 23:05:57.878453 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.878434    2556 scope.go:117] "RemoveContainer" containerID="7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472"
Jul 16 23:05:57.878722 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:05:57.878703    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472\": container with ID starting with 7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472 not found: ID does not exist" containerID="7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472"
Jul 16 23:05:57.878769 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.878732    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472"} err="failed to get container status \"7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472\": rpc error: code = NotFound desc = could not find container \"7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472\": container with ID starting with 7c7b934d1aec93cc812bc7f11d52d5091b80b1d20f423477576ae319c4f12472 not found: ID does not exist"
Jul 16 23:05:57.883027 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.883008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66cnz\" (UniqueName: \"kubernetes.io/projected/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-kube-api-access-66cnz\") pod \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") "
Jul 16 23:05:57.883125 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.883044    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls\") pod \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") "
Jul 16 23:05:57.883171 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.883161    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-openshift-service-ca\") pod \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\" (UID: \"8cd78f60-e7f4-4fdd-90a1-06666cc3dba2\") "
Jul 16 23:05:57.883507 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.883487    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" (UID: "8cd78f60-e7f4-4fdd-90a1-06666cc3dba2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:05:57.885280 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.885256    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-kube-api-access-66cnz" (OuterVolumeSpecName: "kube-api-access-66cnz") pod "8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" (UID: "8cd78f60-e7f4-4fdd-90a1-06666cc3dba2"). InnerVolumeSpecName "kube-api-access-66cnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:05:57.885347 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.885309    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" (UID: "8cd78f60-e7f4-4fdd-90a1-06666cc3dba2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:05:57.984558 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.984522    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:05:57.984558 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.984550    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66cnz\" (UniqueName: \"kubernetes.io/projected/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-kube-api-access-66cnz\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:05:57.984558 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:57.984561    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:05:58.191901 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:58.191867    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"]
Jul 16 23:05:58.198034 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:58.198005    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8c14db-7445bcd76f-7xwwj"]
Jul 16 23:05:59.812453 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:05:59.812402    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" path="/var/lib/kubelet/pods/8cd78f60-e7f4-4fdd-90a1-06666cc3dba2/volumes"
Jul 16 23:06:02.036171 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:02.036136    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:07.804250 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:07.804220    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:07.904767 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:07.904730    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:08.004647 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:08.004614    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:08.066011 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:08.065938    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:12.690327 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:12.690283    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:12.922547 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:12.922506    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:13.261227 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:13.261184    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:13.477851 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:13.477813    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:13.562679 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:13.562647    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:13.642582 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:13.642552    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:13.743303 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:13.743253    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:16.872177 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.872140    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:16.942239 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.942192    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:16.948870 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.948835    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"]
Jul 16 23:06:16.949557 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.949533    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" containerName="maas-api"
Jul 16 23:06:16.949669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.949562    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" containerName="maas-api"
Jul 16 23:06:16.949669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.949577    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="487a32a1-a2e9-4754-a0e8-82818850f895" containerName="maas-api"
Jul 16 23:06:16.949669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.949586    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="487a32a1-a2e9-4754-a0e8-82818850f895" containerName="maas-api"
Jul 16 23:06:16.949837 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.949687    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="487a32a1-a2e9-4754-a0e8-82818850f895" containerName="maas-api"
Jul 16 23:06:16.949837 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.949701    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cd78f60-e7f4-4fdd-90a1-06666cc3dba2" containerName="maas-api"
Jul 16 23:06:16.954774 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.954753    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:16.957176 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.957101    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b5738f-serving-cert\""
Jul 16 23:06:16.957296 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.957239    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5v2t7\""
Jul 16 23:06:16.961314 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:16.961247    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"]
Jul 16 23:06:17.016784 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.016743    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:17.060893 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.060848    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9lzc\" (UniqueName: \"kubernetes.io/projected/f79547d0-f91b-4d14-8f15-1805408088b9-kube-api-access-j9lzc\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.061093 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.060992    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79547d0-f91b-4d14-8f15-1805408088b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.061093 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.061024    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.161633 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.161533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79547d0-f91b-4d14-8f15-1805408088b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.161633 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.161581    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.161633 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.161634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9lzc\" (UniqueName: \"kubernetes.io/projected/f79547d0-f91b-4d14-8f15-1805408088b9-kube-api-access-j9lzc\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.161934 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:06:17.161769    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-serving-cert: secret "maas-api-e2e-shared-a-b5738f-serving-cert" not found
Jul 16 23:06:17.161934 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:06:17.161837    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls podName:f79547d0-f91b-4d14-8f15-1805408088b9 nodeName:}" failed. No retries permitted until 2026-07-16 23:06:17.661817727 +0000 UTC m=+2970.335076724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls") pod "maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm" (UID: "f79547d0-f91b-4d14-8f15-1805408088b9") : secret "maas-api-e2e-shared-a-b5738f-serving-cert" not found
Jul 16 23:06:17.162382 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.162360    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79547d0-f91b-4d14-8f15-1805408088b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.180236 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.180202    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9lzc\" (UniqueName: \"kubernetes.io/projected/f79547d0-f91b-4d14-8f15-1805408088b9-kube-api-access-j9lzc\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.555809 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.555775    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:17.657067 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.657034    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:17.667370 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.667342    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.670268 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.670237    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:17.743982 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.743949    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:17.871762 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:17.871663    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:18.043397 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:18.043358    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"]
Jul 16 23:06:18.047491 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:06:18.047450    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79547d0_f91b_4d14_8f15_1805408088b9.slice/crio-e443ebc6ff52a943cb693ae2a5bfc57656e5d1dbac165d64c6bdd7879e00f097 WatchSource:0}: Error finding container e443ebc6ff52a943cb693ae2a5bfc57656e5d1dbac165d64c6bdd7879e00f097: Status 404 returned error can't find the container with id e443ebc6ff52a943cb693ae2a5bfc57656e5d1dbac165d64c6bdd7879e00f097
Jul 16 23:06:18.999418 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:18.999378    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm" event={"ID":"f79547d0-f91b-4d14-8f15-1805408088b9","Type":"ContainerStarted","Data":"48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b"}
Jul 16 23:06:18.999574 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:18.999441    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:18.999574 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:18.999464    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm" event={"ID":"f79547d0-f91b-4d14-8f15-1805408088b9","Type":"ContainerStarted","Data":"e443ebc6ff52a943cb693ae2a5bfc57656e5d1dbac165d64c6bdd7879e00f097"}
Jul 16 23:06:19.017299 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:19.017233    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm" podStartSLOduration=2.606058504 podStartE2EDuration="3.017210312s" totalImagesPullingTime="411.151808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:06:19.01539975 +0000 UTC m=+2971.688658758" watchObservedRunningTime="2026-07-16 23:06:19.017210312 +0000 UTC m=+2971.690469320"
Jul 16 23:06:21.368876 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:21.368834    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:21.686883 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:21.686796    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:21.998621 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:21.998527    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:22.148734 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:22.148701    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:24.941861 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:24.941825    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:25.010276 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.010240    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:06:25.031678 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.031647    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"]
Jul 16 23:06:25.034239 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.034205    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:25.035697 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.035672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.038011 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.037982    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-61f04e-serving-cert\""
Jul 16 23:06:25.047980 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.047951    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"]
Jul 16 23:06:25.105215 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.105179    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:25.138453 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.138408    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxpld\" (UniqueName: \"kubernetes.io/projected/c44bad23-f916-4198-abde-b8caa6a0a86a-kube-api-access-cxpld\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.138669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.138481    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.138742 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.138652    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bad23-f916-4198-abde-b8caa6a0a86a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.240286 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.240184    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.240286 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.240252    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bad23-f916-4198-abde-b8caa6a0a86a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.240489 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:06:25.240358    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-serving-cert: secret "maas-api-e2e-shared-b-61f04e-serving-cert" not found
Jul 16 23:06:25.240489 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:06:25.240459    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls podName:c44bad23-f916-4198-abde-b8caa6a0a86a nodeName:}" failed. No retries permitted until 2026-07-16 23:06:25.740436393 +0000 UTC m=+2978.413695383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls") pod "maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm" (UID: "c44bad23-f916-4198-abde-b8caa6a0a86a") : secret "maas-api-e2e-shared-b-61f04e-serving-cert" not found
Jul 16 23:06:25.240489 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.240362    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxpld\" (UniqueName: \"kubernetes.io/projected/c44bad23-f916-4198-abde-b8caa6a0a86a-kube-api-access-cxpld\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.240970 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.240951    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bad23-f916-4198-abde-b8caa6a0a86a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.252557 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.252523    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxpld\" (UniqueName: \"kubernetes.io/projected/c44bad23-f916-4198-abde-b8caa6a0a86a-kube-api-access-cxpld\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.529287 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.529191    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:25.689935 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.689898    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:25.745295 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.745258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.748088 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.748067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls\") pod \"maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:25.785731 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.785705    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:25.948330 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:25.948277    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:26.105969 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:26.105935    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"]
Jul 16 23:06:26.107916 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:06:26.107881    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44bad23_f916_4198_abde_b8caa6a0a86a.slice/crio-b1418a4f061c35f893f287129aa8867ee6ab1f0368286805ce8691fc0062f8e3 WatchSource:0}: Error finding container b1418a4f061c35f893f287129aa8867ee6ab1f0368286805ce8691fc0062f8e3: Status 404 returned error can't find the container with id b1418a4f061c35f893f287129aa8867ee6ab1f0368286805ce8691fc0062f8e3
Jul 16 23:06:27.043314 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:27.043260    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm" event={"ID":"c44bad23-f916-4198-abde-b8caa6a0a86a","Type":"ContainerStarted","Data":"e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe"}
Jul 16 23:06:27.043721 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:27.043342    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:27.043721 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:27.043367    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm" event={"ID":"c44bad23-f916-4198-abde-b8caa6a0a86a","Type":"ContainerStarted","Data":"b1418a4f061c35f893f287129aa8867ee6ab1f0368286805ce8691fc0062f8e3"}
Jul 16 23:06:27.064299 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:27.064241    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm" podStartSLOduration=1.737578372 podStartE2EDuration="2.064224006s" totalImagesPullingTime="326.645634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:06:27.062310706 +0000 UTC m=+2979.735569740" watchObservedRunningTime="2026-07-16 23:06:27.064224006 +0000 UTC m=+2979.737483015"
Jul 16 23:06:29.406612 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:29.406579    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:33.055291 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:33.055260    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:06:38.408447 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:38.408389    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:38.506097 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:38.506063    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:41.388079 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:41.388040    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:46.622921 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:46.622877    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:46.722511 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:46.722459    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:06:48.017157 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:48.017097    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 23:06:48.021497 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:48.021473    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 23:06:48.023471 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:48.023445    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 23:06:48.027786 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:06:48.027766    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 23:07:10.667172 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:10.667136    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:07:10.764832 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:10.764797    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:07:25.478447 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.478411    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"]
Jul 16 23:07:25.482266 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.482245    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.484697 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.484665    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-61f04e\"/\"default-dockercfg-rc8hh\""
Jul 16 23:07:25.484697 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.484681    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-61f04e\"/\"openshift-service-ca.crt\""
Jul 16 23:07:25.485623 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.485600    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-61f04e\"/\"auth-test-model-61f04e-kserve-self-signed-certs\""
Jul 16 23:07:25.485736 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.485710    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-61f04e\"/\"kube-root-ca.crt\""
Jul 16 23:07:25.493561 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.493538    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"]
Jul 16 23:07:25.508519 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.508485    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:07:25.590656 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.590621    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-home\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.590656 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.590661    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-dshm\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.590934 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.590743    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-model-cache\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.590934 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.590817    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tmp-dir\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.590934 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.590847    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6mls\" (UniqueName: \"kubernetes.io/projected/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-kube-api-access-r6mls\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.590934 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.590919    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tls-certs\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692448 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692396    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-home\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692448 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-dshm\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692713 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-model-cache\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692713 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692538    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tmp-dir\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692713 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692564    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6mls\" (UniqueName: \"kubernetes.io/projected/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-kube-api-access-r6mls\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692713 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tls-certs\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692920 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692875    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-home\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.692920 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692906    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-model-cache\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.693035 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.692983    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tmp-dir\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.694991 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.694964    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-dshm\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.695349 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.695330    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tls-certs\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.701400 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.701376    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6mls\" (UniqueName: \"kubernetes.io/projected/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-kube-api-access-r6mls\") pod \"auth-test-model-61f04e-kserve-56f4889c95-8kr99\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") " pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.793999 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.793956    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:25.944221 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:25.944072    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"]
Jul 16 23:07:25.947408 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:07:25.947367    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5658a5c_fcb3_4ce0_8c4e_856d4640f613.slice/crio-714f083afe7c37470d36e2141fb9b2dcc43690b31eb10ce99c421e3c4cc1fe69 WatchSource:0}: Error finding container 714f083afe7c37470d36e2141fb9b2dcc43690b31eb10ce99c421e3c4cc1fe69: Status 404 returned error can't find the container with id 714f083afe7c37470d36e2141fb9b2dcc43690b31eb10ce99c421e3c4cc1fe69
Jul 16 23:07:26.349222 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:26.349185    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99" event={"ID":"b5658a5c-fcb3-4ce0-8c4e-856d4640f613","Type":"ContainerStarted","Data":"4bb17067d53305a400a1c74386e83cecedaa33eac245b13d48c3be2f06a59b7d"}
Jul 16 23:07:26.349222 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:26.349229    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99" event={"ID":"b5658a5c-fcb3-4ce0-8c4e-856d4640f613","Type":"ContainerStarted","Data":"714f083afe7c37470d36e2141fb9b2dcc43690b31eb10ce99c421e3c4cc1fe69"}
Jul 16 23:07:26.365643 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:26.365579    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99" podStartSLOduration=1.3655585160000001 podStartE2EDuration="1.365558516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:07:26.364483246 +0000 UTC m=+3039.037742269" watchObservedRunningTime="2026-07-16 23:07:26.365558516 +0000 UTC m=+3039.038817526"
Jul 16 23:07:35.794754 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:35.794714    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:35.795248 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:35.794768    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:35.812103 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:35.812076    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:36.421055 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:36.421016    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:07:45.017403 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:45.017325    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:07:45.117089 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:07:45.117052    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:09:25.760633 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:25.760601    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:09:26.787277 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:26.787244    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"]
Jul 16 23:09:26.805481 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:26.805445    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:09:32.898370 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:32.898337    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:09:41.809604 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:41.809554    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99" podUID="b5658a5c-fcb3-4ce0-8c4e-856d4640f613" containerName="main" containerID="cri-o://4bb17067d53305a400a1c74386e83cecedaa33eac245b13d48c3be2f06a59b7d" gracePeriod=45
Jul 16 23:09:42.064296 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.064266    2556 generic.go:356] "Generic (PLEG): container finished" podID="b5658a5c-fcb3-4ce0-8c4e-856d4640f613" containerID="4bb17067d53305a400a1c74386e83cecedaa33eac245b13d48c3be2f06a59b7d" exitCode=0
Jul 16 23:09:42.064432 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.064339    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99" event={"ID":"b5658a5c-fcb3-4ce0-8c4e-856d4640f613","Type":"ContainerDied","Data":"4bb17067d53305a400a1c74386e83cecedaa33eac245b13d48c3be2f06a59b7d"}
Jul 16 23:09:42.079200 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.079175    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:09:42.131920 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.131895    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-home\") pod \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") "
Jul 16 23:09:42.132074 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.131986    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tmp-dir\") pod \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") "
Jul 16 23:09:42.132074 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132015    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tls-certs\") pod \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") "
Jul 16 23:09:42.132074 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132034    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6mls\" (UniqueName: \"kubernetes.io/projected/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-kube-api-access-r6mls\") pod \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") "
Jul 16 23:09:42.132074 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132055    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-dshm\") pod \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") "
Jul 16 23:09:42.132074 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132070    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-model-cache\") pod \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\" (UID: \"b5658a5c-fcb3-4ce0-8c4e-856d4640f613\") "
Jul 16 23:09:42.132357 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132147    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-home" (OuterVolumeSpecName: "home") pod "b5658a5c-fcb3-4ce0-8c4e-856d4640f613" (UID: "b5658a5c-fcb3-4ce0-8c4e-856d4640f613"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:09:42.132423 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132404    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-home\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:09:42.132510 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132450    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5658a5c-fcb3-4ce0-8c4e-856d4640f613" (UID: "b5658a5c-fcb3-4ce0-8c4e-856d4640f613"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:09:42.132562 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.132479    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5658a5c-fcb3-4ce0-8c4e-856d4640f613" (UID: "b5658a5c-fcb3-4ce0-8c4e-856d4640f613"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:09:42.134514 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.134490    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-dshm" (OuterVolumeSpecName: "dshm") pod "b5658a5c-fcb3-4ce0-8c4e-856d4640f613" (UID: "b5658a5c-fcb3-4ce0-8c4e-856d4640f613"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:09:42.134596 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.134533    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5658a5c-fcb3-4ce0-8c4e-856d4640f613" (UID: "b5658a5c-fcb3-4ce0-8c4e-856d4640f613"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:09:42.134639 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.134608    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-kube-api-access-r6mls" (OuterVolumeSpecName: "kube-api-access-r6mls") pod "b5658a5c-fcb3-4ce0-8c4e-856d4640f613" (UID: "b5658a5c-fcb3-4ce0-8c4e-856d4640f613"). InnerVolumeSpecName "kube-api-access-r6mls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:09:42.233340 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.233305    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tmp-dir\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:09:42.233340 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.233334    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-tls-certs\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:09:42.233340 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.233346    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6mls\" (UniqueName: \"kubernetes.io/projected/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-kube-api-access-r6mls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:09:42.233541 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.233359    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-dshm\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:09:42.233541 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:42.233371    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5658a5c-fcb3-4ce0-8c4e-856d4640f613-model-cache\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:09:43.071231 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:43.071200    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"
Jul 16 23:09:43.071642 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:43.071232    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99" event={"ID":"b5658a5c-fcb3-4ce0-8c4e-856d4640f613","Type":"ContainerDied","Data":"714f083afe7c37470d36e2141fb9b2dcc43690b31eb10ce99c421e3c4cc1fe69"}
Jul 16 23:09:43.071642 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:43.071283    2556 scope.go:117] "RemoveContainer" containerID="4bb17067d53305a400a1c74386e83cecedaa33eac245b13d48c3be2f06a59b7d"
Jul 16 23:09:43.092660 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:43.092632    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"]
Jul 16 23:09:43.095102 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:43.095079    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-61f04e/auth-test-model-61f04e-kserve-56f4889c95-8kr99"]
Jul 16 23:09:43.814005 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:43.813972    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5658a5c-fcb3-4ce0-8c4e-856d4640f613" path="/var/lib/kubelet/pods/b5658a5c-fcb3-4ce0-8c4e-856d4640f613/volumes"
Jul 16 23:09:53.166558 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:53.166521    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:09:59.465530 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:09:59.465493    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:10:16.129018 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:10:16.128972    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:10:16.229274 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:10:16.229231    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:10:28.662256 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:10:28.662200    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:10:29.474022 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:10:29.473992    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:10:41.902834 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:10:41.902736    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:14.230570 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:14.230529    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:15.337977 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:15.337887    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:19.410574 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:19.410521    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:25.360897 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:25.360860    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:39.146685 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:39.146648    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:39.246155 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:39.246089    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:48.042209 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:48.042166    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:48.051730 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:48.051710    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 23:11:48.056522 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:48.056497    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 23:11:48.059077 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:48.059057    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 23:11:48.063268 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:48.063250    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8k4z4_622ef6b4-0643-4162-b807-27de4bcd3ffd/ovn-acl-logging/0.log"
Jul 16 23:11:48.100545 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:48.100507    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:53.470542 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:53.470501    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:11:53.524683 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:11:53.524646    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:07.979888 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:07.979850    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:08.791925 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:08.791841    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:09.706413 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.706379    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7525-140274772261680"]
Jul 16 23:12:09.706848 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.706835    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5658a5c-fcb3-4ce0-8c4e-856d4640f613" containerName="main"
Jul 16 23:12:09.706896 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.706850    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5658a5c-fcb3-4ce0-8c4e-856d4640f613" containerName="main"
Jul 16 23:12:09.706940 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.706931    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5658a5c-fcb3-4ce0-8c4e-856d4640f613" containerName="main"
Jul 16 23:12:09.710021 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.710004    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772261680"
Jul 16 23:12:09.712352 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.712328    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jzfmx\""
Jul 16 23:12:09.716675 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.716655    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772261680"]
Jul 16 23:12:09.777975 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.777948    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxm65\" (UniqueName: \"kubernetes.io/projected/d06e73a6-e081-4f4c-b935-6f3dd9ad2a66-kube-api-access-cxm65\") pod \"test-curl-7525-140274772261680\" (UID: \"d06e73a6-e081-4f4c-b935-6f3dd9ad2a66\") " pod="opendatahub/test-curl-7525-140274772261680"
Jul 16 23:12:09.879510 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.879473    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxm65\" (UniqueName: \"kubernetes.io/projected/d06e73a6-e081-4f4c-b935-6f3dd9ad2a66-kube-api-access-cxm65\") pod \"test-curl-7525-140274772261680\" (UID: \"d06e73a6-e081-4f4c-b935-6f3dd9ad2a66\") " pod="opendatahub/test-curl-7525-140274772261680"
Jul 16 23:12:09.887699 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:09.887671    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxm65\" (UniqueName: \"kubernetes.io/projected/d06e73a6-e081-4f4c-b935-6f3dd9ad2a66-kube-api-access-cxm65\") pod \"test-curl-7525-140274772261680\" (UID: \"d06e73a6-e081-4f4c-b935-6f3dd9ad2a66\") " pod="opendatahub/test-curl-7525-140274772261680"
Jul 16 23:12:10.020351 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:10.020268    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772261680"
Jul 16 23:12:10.162648 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:10.162387    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772261680"]
Jul 16 23:12:10.166437 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:12:10.166404    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd06e73a6_e081_4f4c_b935_6f3dd9ad2a66.slice/crio-d8d1997ba20b627797f0045d4dcc1009dd4c090483697101ef77dd1bf7a0464f WatchSource:0}: Error finding container d8d1997ba20b627797f0045d4dcc1009dd4c090483697101ef77dd1bf7a0464f: Status 404 returned error can't find the container with id d8d1997ba20b627797f0045d4dcc1009dd4c090483697101ef77dd1bf7a0464f
Jul 16 23:12:10.168258 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:10.168239    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 23:12:10.812545 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:10.812511    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772261680" event={"ID":"d06e73a6-e081-4f4c-b935-6f3dd9ad2a66","Type":"ContainerStarted","Data":"d8d1997ba20b627797f0045d4dcc1009dd4c090483697101ef77dd1bf7a0464f"}
Jul 16 23:12:11.819026 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:11.818989    2556 generic.go:356] "Generic (PLEG): container finished" podID="d06e73a6-e081-4f4c-b935-6f3dd9ad2a66" containerID="b0f0234a0686a9c7ac926398f9171528ebbd5f7052466895c3caba0b93531b80" exitCode=0
Jul 16 23:12:11.819421 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:11.819046    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772261680" event={"ID":"d06e73a6-e081-4f4c-b935-6f3dd9ad2a66","Type":"ContainerDied","Data":"b0f0234a0686a9c7ac926398f9171528ebbd5f7052466895c3caba0b93531b80"}
Jul 16 23:12:12.965129 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:12.965074    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772261680"
Jul 16 23:12:12.985985 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:12.985956    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7525-140274772261680_d06e73a6-e081-4f4c-b935-6f3dd9ad2a66/test-curl-7525-140274772261680/0.log"
Jul 16 23:12:13.007902 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.007875    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7525-140274772261680"]
Jul 16 23:12:13.011978 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.011952    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7525-140274772261680"]
Jul 16 23:12:13.108347 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.108263    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxm65\" (UniqueName: \"kubernetes.io/projected/d06e73a6-e081-4f4c-b935-6f3dd9ad2a66-kube-api-access-cxm65\") pod \"d06e73a6-e081-4f4c-b935-6f3dd9ad2a66\" (UID: \"d06e73a6-e081-4f4c-b935-6f3dd9ad2a66\") "
Jul 16 23:12:13.110606 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.110571    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06e73a6-e081-4f4c-b935-6f3dd9ad2a66-kube-api-access-cxm65" (OuterVolumeSpecName: "kube-api-access-cxm65") pod "d06e73a6-e081-4f4c-b935-6f3dd9ad2a66" (UID: "d06e73a6-e081-4f4c-b935-6f3dd9ad2a66"). InnerVolumeSpecName "kube-api-access-cxm65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:13.131218 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.131186    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7525-140274772389936"]
Jul 16 23:12:13.131621 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.131605    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06e73a6-e081-4f4c-b935-6f3dd9ad2a66" containerName="test-curl-7525-140274772261680"
Jul 16 23:12:13.131621 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.131620    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06e73a6-e081-4f4c-b935-6f3dd9ad2a66" containerName="test-curl-7525-140274772261680"
Jul 16 23:12:13.131756 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.131690    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06e73a6-e081-4f4c-b935-6f3dd9ad2a66" containerName="test-curl-7525-140274772261680"
Jul 16 23:12:13.134632 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.134612    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:13.141253 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.141228    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772389936"]
Jul 16 23:12:13.209288 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.209254    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhhck\" (UniqueName: \"kubernetes.io/projected/25663ce9-fe83-4662-9be6-229f451d3547-kube-api-access-hhhck\") pod \"test-curl-7525-140274772389936\" (UID: \"25663ce9-fe83-4662-9be6-229f451d3547\") " pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:13.209433 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.209386    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxm65\" (UniqueName: \"kubernetes.io/projected/d06e73a6-e081-4f4c-b935-6f3dd9ad2a66-kube-api-access-cxm65\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:13.310530 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.310495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhhck\" (UniqueName: \"kubernetes.io/projected/25663ce9-fe83-4662-9be6-229f451d3547-kube-api-access-hhhck\") pod \"test-curl-7525-140274772389936\" (UID: \"25663ce9-fe83-4662-9be6-229f451d3547\") " pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:13.318522 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.318493    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhhck\" (UniqueName: \"kubernetes.io/projected/25663ce9-fe83-4662-9be6-229f451d3547-kube-api-access-hhhck\") pod \"test-curl-7525-140274772389936\" (UID: \"25663ce9-fe83-4662-9be6-229f451d3547\") " pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:13.445714 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.445632    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:13.592551 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.592515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772389936"]
Jul 16 23:12:13.595801 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:12:13.595779    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25663ce9_fe83_4662_9be6_229f451d3547.slice/crio-bdd85e8c609e9dbd2217d33bcbea7d856dd1b282021750a696b042ff76c0d1b0 WatchSource:0}: Error finding container bdd85e8c609e9dbd2217d33bcbea7d856dd1b282021750a696b042ff76c0d1b0: Status 404 returned error can't find the container with id bdd85e8c609e9dbd2217d33bcbea7d856dd1b282021750a696b042ff76c0d1b0
Jul 16 23:12:13.812649 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.812621    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d06e73a6-e081-4f4c-b935-6f3dd9ad2a66" path="/var/lib/kubelet/pods/d06e73a6-e081-4f4c-b935-6f3dd9ad2a66/volumes"
Jul 16 23:12:13.831120 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.831081    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772261680"
Jul 16 23:12:13.831241 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.831158    2556 scope.go:117] "RemoveContainer" containerID="b0f0234a0686a9c7ac926398f9171528ebbd5f7052466895c3caba0b93531b80"
Jul 16 23:12:13.832474 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:13.832252    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772389936" event={"ID":"25663ce9-fe83-4662-9be6-229f451d3547","Type":"ContainerStarted","Data":"bdd85e8c609e9dbd2217d33bcbea7d856dd1b282021750a696b042ff76c0d1b0"}
Jul 16 23:12:14.839459 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:14.839415    2556 generic.go:356] "Generic (PLEG): container finished" podID="25663ce9-fe83-4662-9be6-229f451d3547" containerID="b8971bb46ec9ae295491d6228e06b4168d013a2d5d07b4c493de6c5851d55538" exitCode=0
Jul 16 23:12:14.839873 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:14.839476    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772389936" event={"ID":"25663ce9-fe83-4662-9be6-229f451d3547","Type":"ContainerDied","Data":"b8971bb46ec9ae295491d6228e06b4168d013a2d5d07b4c493de6c5851d55538"}
Jul 16 23:12:15.991950 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:15.991925    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:16.012647 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.012602    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7525-140274772389936_25663ce9-fe83-4662-9be6-229f451d3547/test-curl-7525-140274772389936/0.log"
Jul 16 23:12:16.040622 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.040587    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7525-140274772389936"]
Jul 16 23:12:16.043480 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.043444    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7525-140274772389936"]
Jul 16 23:12:16.139635 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.139549    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhhck\" (UniqueName: \"kubernetes.io/projected/25663ce9-fe83-4662-9be6-229f451d3547-kube-api-access-hhhck\") pod \"25663ce9-fe83-4662-9be6-229f451d3547\" (UID: \"25663ce9-fe83-4662-9be6-229f451d3547\") "
Jul 16 23:12:16.141893 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.141863    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25663ce9-fe83-4662-9be6-229f451d3547-kube-api-access-hhhck" (OuterVolumeSpecName: "kube-api-access-hhhck") pod "25663ce9-fe83-4662-9be6-229f451d3547" (UID: "25663ce9-fe83-4662-9be6-229f451d3547"). InnerVolumeSpecName "kube-api-access-hhhck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:16.197613 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.197585    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:16.198025 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.198012    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25663ce9-fe83-4662-9be6-229f451d3547" containerName="test-curl-7525-140274772389936"
Jul 16 23:12:16.198067 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.198026    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="25663ce9-fe83-4662-9be6-229f451d3547" containerName="test-curl-7525-140274772389936"
Jul 16 23:12:16.198125 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.198102    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="25663ce9-fe83-4662-9be6-229f451d3547" containerName="test-curl-7525-140274772389936"
Jul 16 23:12:16.201252 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.201235    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:16.206868 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.206845    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:16.240640 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.240611    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhhck\" (UniqueName: \"kubernetes.io/projected/25663ce9-fe83-4662-9be6-229f451d3547-kube-api-access-hhhck\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:16.341512 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.341478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crnlq\" (UniqueName: \"kubernetes.io/projected/717aff58-6539-4a5f-91e9-97a03bf0cf68-kube-api-access-crnlq\") pod \"test-curl-7525-140274772276544\" (UID: \"717aff58-6539-4a5f-91e9-97a03bf0cf68\") " pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:16.442235 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.442162    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crnlq\" (UniqueName: \"kubernetes.io/projected/717aff58-6539-4a5f-91e9-97a03bf0cf68-kube-api-access-crnlq\") pod \"test-curl-7525-140274772276544\" (UID: \"717aff58-6539-4a5f-91e9-97a03bf0cf68\") " pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:16.450147 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.450124    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crnlq\" (UniqueName: \"kubernetes.io/projected/717aff58-6539-4a5f-91e9-97a03bf0cf68-kube-api-access-crnlq\") pod \"test-curl-7525-140274772276544\" (UID: \"717aff58-6539-4a5f-91e9-97a03bf0cf68\") " pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:16.512118 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.512097    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:16.651652 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.651628    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:16.653665 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:12:16.653638    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod717aff58_6539_4a5f_91e9_97a03bf0cf68.slice/crio-defc1405612e8dd739ee26bce4eadd0ed09f27bec76707c379cde4269d153127 WatchSource:0}: Error finding container defc1405612e8dd739ee26bce4eadd0ed09f27bec76707c379cde4269d153127: Status 404 returned error can't find the container with id defc1405612e8dd739ee26bce4eadd0ed09f27bec76707c379cde4269d153127
Jul 16 23:12:16.851445 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.851407    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772276544" event={"ID":"717aff58-6539-4a5f-91e9-97a03bf0cf68","Type":"ContainerStarted","Data":"defc1405612e8dd739ee26bce4eadd0ed09f27bec76707c379cde4269d153127"}
Jul 16 23:12:16.852715 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.852694    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdd85e8c609e9dbd2217d33bcbea7d856dd1b282021750a696b042ff76c0d1b0"
Jul 16 23:12:16.852715 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:16.852703    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:16.863565 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:12:16.863538    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7525-140274772389936\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="25663ce9-fe83-4662-9be6-229f451d3547" pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:17.810495 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:12:17.810460    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7525-140274772389936\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="25663ce9-fe83-4662-9be6-229f451d3547" pod="opendatahub/test-curl-7525-140274772389936"
Jul 16 23:12:17.812435 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:17.812413    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25663ce9-fe83-4662-9be6-229f451d3547" path="/var/lib/kubelet/pods/25663ce9-fe83-4662-9be6-229f451d3547/volumes"
Jul 16 23:12:17.859272 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:17.859242    2556 generic.go:356] "Generic (PLEG): container finished" podID="717aff58-6539-4a5f-91e9-97a03bf0cf68" containerID="96ad2c7ff063c737e9959d69f2b4582d90d6ee998efc56e20d5de480d22dc9a2" exitCode=0
Jul 16 23:12:17.859384 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:17.859285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772276544" event={"ID":"717aff58-6539-4a5f-91e9-97a03bf0cf68","Type":"ContainerDied","Data":"96ad2c7ff063c737e9959d69f2b4582d90d6ee998efc56e20d5de480d22dc9a2"}
Jul 16 23:12:19.017575 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.017548    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:19.038855 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.038830    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7525-140274772276544_717aff58-6539-4a5f-91e9-97a03bf0cf68/test-curl-7525-140274772276544/0.log"
Jul 16 23:12:19.062388 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.062360    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:19.066697 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.066673    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:19.166178 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.166083    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crnlq\" (UniqueName: \"kubernetes.io/projected/717aff58-6539-4a5f-91e9-97a03bf0cf68-kube-api-access-crnlq\") pod \"717aff58-6539-4a5f-91e9-97a03bf0cf68\" (UID: \"717aff58-6539-4a5f-91e9-97a03bf0cf68\") "
Jul 16 23:12:19.168264 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.168236    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/717aff58-6539-4a5f-91e9-97a03bf0cf68-kube-api-access-crnlq" (OuterVolumeSpecName: "kube-api-access-crnlq") pod "717aff58-6539-4a5f-91e9-97a03bf0cf68" (UID: "717aff58-6539-4a5f-91e9-97a03bf0cf68"). InnerVolumeSpecName "kube-api-access-crnlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:19.180856 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.180834    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:19.181246 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.181233    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="717aff58-6539-4a5f-91e9-97a03bf0cf68" containerName="test-curl-7525-140274772276544"
Jul 16 23:12:19.181286 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.181247    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="717aff58-6539-4a5f-91e9-97a03bf0cf68" containerName="test-curl-7525-140274772276544"
Jul 16 23:12:19.181339 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.181330    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="717aff58-6539-4a5f-91e9-97a03bf0cf68" containerName="test-curl-7525-140274772276544"
Jul 16 23:12:19.184282 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.184267    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:19.190781 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.190757    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:19.267263 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.267236    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlbj\" (UniqueName: \"kubernetes.io/projected/8dbd92c3-d379-460a-b1aa-21f06a12bc0a-kube-api-access-gdlbj\") pod \"test-curl-7525-140274772276544\" (UID: \"8dbd92c3-d379-460a-b1aa-21f06a12bc0a\") " pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:19.267407 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.267310    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crnlq\" (UniqueName: \"kubernetes.io/projected/717aff58-6539-4a5f-91e9-97a03bf0cf68-kube-api-access-crnlq\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:19.368030 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.367994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlbj\" (UniqueName: \"kubernetes.io/projected/8dbd92c3-d379-460a-b1aa-21f06a12bc0a-kube-api-access-gdlbj\") pod \"test-curl-7525-140274772276544\" (UID: \"8dbd92c3-d379-460a-b1aa-21f06a12bc0a\") " pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:19.376279 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.376248    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlbj\" (UniqueName: \"kubernetes.io/projected/8dbd92c3-d379-460a-b1aa-21f06a12bc0a-kube-api-access-gdlbj\") pod \"test-curl-7525-140274772276544\" (UID: \"8dbd92c3-d379-460a-b1aa-21f06a12bc0a\") " pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:19.496101 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.496024    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:19.632986 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.632950    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:19.636073 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:12:19.636040    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dbd92c3_d379_460a_b1aa_21f06a12bc0a.slice/crio-afdf72bc0e9ca6c63e79a7c0377befb32712415468d1daececcdc0c1387ceeee WatchSource:0}: Error finding container afdf72bc0e9ca6c63e79a7c0377befb32712415468d1daececcdc0c1387ceeee: Status 404 returned error can't find the container with id afdf72bc0e9ca6c63e79a7c0377befb32712415468d1daececcdc0c1387ceeee
Jul 16 23:12:19.812025 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.811990    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="717aff58-6539-4a5f-91e9-97a03bf0cf68" path="/var/lib/kubelet/pods/717aff58-6539-4a5f-91e9-97a03bf0cf68/volumes"
Jul 16 23:12:19.871170 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.871137    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:19.871342 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.871137    2556 scope.go:117] "RemoveContainer" containerID="96ad2c7ff063c737e9959d69f2b4582d90d6ee998efc56e20d5de480d22dc9a2"
Jul 16 23:12:19.872637 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:19.872387    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772276544" event={"ID":"8dbd92c3-d379-460a-b1aa-21f06a12bc0a","Type":"ContainerStarted","Data":"afdf72bc0e9ca6c63e79a7c0377befb32712415468d1daececcdc0c1387ceeee"}
Jul 16 23:12:20.880209 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:20.880169    2556 generic.go:356] "Generic (PLEG): container finished" podID="8dbd92c3-d379-460a-b1aa-21f06a12bc0a" containerID="02876889bf05b5880a921b432941c8c85fd2c9ac8fe92ec9f63b0b217fdd5758" exitCode=0
Jul 16 23:12:20.880682 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:20.880257    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772276544" event={"ID":"8dbd92c3-d379-460a-b1aa-21f06a12bc0a","Type":"ContainerDied","Data":"02876889bf05b5880a921b432941c8c85fd2c9ac8fe92ec9f63b0b217fdd5758"}
Jul 16 23:12:22.027174 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.027148    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:22.045829 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.045788    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7525-140274772276544_8dbd92c3-d379-460a-b1aa-21f06a12bc0a/test-curl-7525-140274772276544/0.log"
Jul 16 23:12:22.068223 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.068190    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:22.070167 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.070142    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7525-140274772276544"]
Jul 16 23:12:22.187946 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.187871    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7525-140274772277552"]
Jul 16 23:12:22.188292 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.188279    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dbd92c3-d379-460a-b1aa-21f06a12bc0a" containerName="test-curl-7525-140274772276544"
Jul 16 23:12:22.188340 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.188294    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dbd92c3-d379-460a-b1aa-21f06a12bc0a" containerName="test-curl-7525-140274772276544"
Jul 16 23:12:22.188392 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.188382    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dbd92c3-d379-460a-b1aa-21f06a12bc0a" containerName="test-curl-7525-140274772276544"
Jul 16 23:12:22.190669 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.190645    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdlbj\" (UniqueName: \"kubernetes.io/projected/8dbd92c3-d379-460a-b1aa-21f06a12bc0a-kube-api-access-gdlbj\") pod \"8dbd92c3-d379-460a-b1aa-21f06a12bc0a\" (UID: \"8dbd92c3-d379-460a-b1aa-21f06a12bc0a\") "
Jul 16 23:12:22.191481 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.191461    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772277552"
Jul 16 23:12:22.193246 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.193215    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbd92c3-d379-460a-b1aa-21f06a12bc0a-kube-api-access-gdlbj" (OuterVolumeSpecName: "kube-api-access-gdlbj") pod "8dbd92c3-d379-460a-b1aa-21f06a12bc0a" (UID: "8dbd92c3-d379-460a-b1aa-21f06a12bc0a"). InnerVolumeSpecName "kube-api-access-gdlbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:22.195641 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.195618    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772277552"]
Jul 16 23:12:22.291933 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.291908    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv46f\" (UniqueName: \"kubernetes.io/projected/d51feb54-74d8-4144-86ba-322eb524f812-kube-api-access-vv46f\") pod \"test-curl-7525-140274772277552\" (UID: \"d51feb54-74d8-4144-86ba-322eb524f812\") " pod="opendatahub/test-curl-7525-140274772277552"
Jul 16 23:12:22.292058 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.291964    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdlbj\" (UniqueName: \"kubernetes.io/projected/8dbd92c3-d379-460a-b1aa-21f06a12bc0a-kube-api-access-gdlbj\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:22.393017 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.392988    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv46f\" (UniqueName: \"kubernetes.io/projected/d51feb54-74d8-4144-86ba-322eb524f812-kube-api-access-vv46f\") pod \"test-curl-7525-140274772277552\" (UID: \"d51feb54-74d8-4144-86ba-322eb524f812\") " pod="opendatahub/test-curl-7525-140274772277552"
Jul 16 23:12:22.401091 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.401061    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv46f\" (UniqueName: \"kubernetes.io/projected/d51feb54-74d8-4144-86ba-322eb524f812-kube-api-access-vv46f\") pod \"test-curl-7525-140274772277552\" (UID: \"d51feb54-74d8-4144-86ba-322eb524f812\") " pod="opendatahub/test-curl-7525-140274772277552"
Jul 16 23:12:22.510705 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.510634    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772277552"
Jul 16 23:12:22.648875 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.648847    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7525-140274772277552"]
Jul 16 23:12:22.650055 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:12:22.650015    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd51feb54_74d8_4144_86ba_322eb524f812.slice/crio-f50ea2b8b4f8a660ab169af0713d7667efe3ce48d61421c21bc8ab2756df1934 WatchSource:0}: Error finding container f50ea2b8b4f8a660ab169af0713d7667efe3ce48d61421c21bc8ab2756df1934: Status 404 returned error can't find the container with id f50ea2b8b4f8a660ab169af0713d7667efe3ce48d61421c21bc8ab2756df1934
Jul 16 23:12:22.891099 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.891059    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772277552" event={"ID":"d51feb54-74d8-4144-86ba-322eb524f812","Type":"ContainerStarted","Data":"f50ea2b8b4f8a660ab169af0713d7667efe3ce48d61421c21bc8ab2756df1934"}
Jul 16 23:12:22.892287 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.892262    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:22.892400 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:22.892290    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afdf72bc0e9ca6c63e79a7c0377befb32712415468d1daececcdc0c1387ceeee"
Jul 16 23:12:22.902181 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:12:22.902156    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7525-140274772276544\" is forbidden: User \"system:node:ip-10-0-139-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-105.ec2.internal' and this object" podUID="8dbd92c3-d379-460a-b1aa-21f06a12bc0a" pod="opendatahub/test-curl-7525-140274772276544"
Jul 16 23:12:23.812081 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:23.812048    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dbd92c3-d379-460a-b1aa-21f06a12bc0a" path="/var/lib/kubelet/pods/8dbd92c3-d379-460a-b1aa-21f06a12bc0a/volumes"
Jul 16 23:12:23.898185 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:23.898152    2556 generic.go:356] "Generic (PLEG): container finished" podID="d51feb54-74d8-4144-86ba-322eb524f812" containerID="20654f9d9b7030c9d91eaac3def8ccef190968012bea18974a103769b46d3af4" exitCode=0
Jul 16 23:12:23.898330 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:23.898234    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7525-140274772277552" event={"ID":"d51feb54-74d8-4144-86ba-322eb524f812","Type":"ContainerDied","Data":"20654f9d9b7030c9d91eaac3def8ccef190968012bea18974a103769b46d3af4"}
Jul 16 23:12:25.050672 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.050646    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772277552"
Jul 16 23:12:25.068913 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.068838    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7525-140274772277552_d51feb54-74d8-4144-86ba-322eb524f812/test-curl-7525-140274772277552/0.log"
Jul 16 23:12:25.091189 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.091142    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7525-140274772277552"]
Jul 16 23:12:25.093066 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.093043    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7525-140274772277552"]
Jul 16 23:12:25.217171 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.217050    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv46f\" (UniqueName: \"kubernetes.io/projected/d51feb54-74d8-4144-86ba-322eb524f812-kube-api-access-vv46f\") pod \"d51feb54-74d8-4144-86ba-322eb524f812\" (UID: \"d51feb54-74d8-4144-86ba-322eb524f812\") "
Jul 16 23:12:25.219330 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.219299    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d51feb54-74d8-4144-86ba-322eb524f812-kube-api-access-vv46f" (OuterVolumeSpecName: "kube-api-access-vv46f") pod "d51feb54-74d8-4144-86ba-322eb524f812" (UID: "d51feb54-74d8-4144-86ba-322eb524f812"). InnerVolumeSpecName "kube-api-access-vv46f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:25.318751 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.318710    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv46f\" (UniqueName: \"kubernetes.io/projected/d51feb54-74d8-4144-86ba-322eb524f812-kube-api-access-vv46f\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:25.811715 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.811679    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d51feb54-74d8-4144-86ba-322eb524f812" path="/var/lib/kubelet/pods/d51feb54-74d8-4144-86ba-322eb524f812/volumes"
Jul 16 23:12:25.916638 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.916607    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7525-140274772277552"
Jul 16 23:12:25.916794 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:25.916582    2556 scope.go:117] "RemoveContainer" containerID="20654f9d9b7030c9d91eaac3def8ccef190968012bea18974a103769b46d3af4"
Jul 16 23:12:28.513477 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:28.513443    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:28.618235 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:28.618203    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:29.719541 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:29.719505    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:38.566228 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.566148    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"]
Jul 16 23:12:38.566755 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.566417    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm" podUID="f79547d0-f91b-4d14-8f15-1805408088b9" containerName="maas-api" containerID="cri-o://48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b" gracePeriod=30
Jul 16 23:12:38.593447 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.593416    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:38.835992 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.835970    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:12:38.840303 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.840284    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls\") pod \"f79547d0-f91b-4d14-8f15-1805408088b9\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") "
Jul 16 23:12:38.840397 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.840353    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9lzc\" (UniqueName: \"kubernetes.io/projected/f79547d0-f91b-4d14-8f15-1805408088b9-kube-api-access-j9lzc\") pod \"f79547d0-f91b-4d14-8f15-1805408088b9\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") "
Jul 16 23:12:38.840457 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.840414    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79547d0-f91b-4d14-8f15-1805408088b9-openshift-service-ca\") pod \"f79547d0-f91b-4d14-8f15-1805408088b9\" (UID: \"f79547d0-f91b-4d14-8f15-1805408088b9\") "
Jul 16 23:12:38.840807 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.840783    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f79547d0-f91b-4d14-8f15-1805408088b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f79547d0-f91b-4d14-8f15-1805408088b9" (UID: "f79547d0-f91b-4d14-8f15-1805408088b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:12:38.842361 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.842338    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f79547d0-f91b-4d14-8f15-1805408088b9" (UID: "f79547d0-f91b-4d14-8f15-1805408088b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:12:38.842741 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.842710    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79547d0-f91b-4d14-8f15-1805408088b9-kube-api-access-j9lzc" (OuterVolumeSpecName: "kube-api-access-j9lzc") pod "f79547d0-f91b-4d14-8f15-1805408088b9" (UID: "f79547d0-f91b-4d14-8f15-1805408088b9"). InnerVolumeSpecName "kube-api-access-j9lzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:12:38.941016 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.940987    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f79547d0-f91b-4d14-8f15-1805408088b9-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:38.941016 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.941013    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9lzc\" (UniqueName: \"kubernetes.io/projected/f79547d0-f91b-4d14-8f15-1805408088b9-kube-api-access-j9lzc\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:38.941218 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.941024    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f79547d0-f91b-4d14-8f15-1805408088b9-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:12:38.984229 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.984198    2556 generic.go:356] "Generic (PLEG): container finished" podID="f79547d0-f91b-4d14-8f15-1805408088b9" containerID="48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b" exitCode=0
Jul 16 23:12:38.984335 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.984242    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"
Jul 16 23:12:38.984335 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.984292    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm" event={"ID":"f79547d0-f91b-4d14-8f15-1805408088b9","Type":"ContainerDied","Data":"48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b"}
Jul 16 23:12:38.984409 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.984343    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm" event={"ID":"f79547d0-f91b-4d14-8f15-1805408088b9","Type":"ContainerDied","Data":"e443ebc6ff52a943cb693ae2a5bfc57656e5d1dbac165d64c6bdd7879e00f097"}
Jul 16 23:12:38.984409 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.984366    2556 scope.go:117] "RemoveContainer" containerID="48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b"
Jul 16 23:12:38.993347 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.993328    2556 scope.go:117] "RemoveContainer" containerID="48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b"
Jul 16 23:12:38.993595 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:12:38.993575    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b\": container with ID starting with 48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b not found: ID does not exist" containerID="48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b"
Jul 16 23:12:38.993640 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:38.993604    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b"} err="failed to get container status \"48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b\": rpc error: code = NotFound desc = could not find container \"48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b\": container with ID starting with 48ae3cbd6c5adf50fc5592d1402ec9bdaea90f289f37a16fa5f68b6ea57f131b not found: ID does not exist"
Jul 16 23:12:39.005334 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:39.005304    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"]
Jul 16 23:12:39.007775 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:39.007755    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5738f-584f9fffb-xrsqm"]
Jul 16 23:12:39.812391 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:39.812356    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f79547d0-f91b-4d14-8f15-1805408088b9" path="/var/lib/kubelet/pods/f79547d0-f91b-4d14-8f15-1805408088b9/volumes"
Jul 16 23:12:43.099203 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:43.099159    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:48.996039 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:48.995996    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:49.096303 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:49.096273    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:53.752274 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:53.752238    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:53.964042 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:53.964005    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:54.597918 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:54.597881    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:12:54.697470 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:12:54.697422    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:04.476131 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:04.476082    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"]
Jul 16 23:13:04.476658 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:04.476408    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm" podUID="c44bad23-f916-4198-abde-b8caa6a0a86a" containerName="maas-api" containerID="cri-o://e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe" gracePeriod=30
Jul 16 23:13:04.511938 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:04.511893    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:04.949929 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:04.949907    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:13:05.062569 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.062537    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxpld\" (UniqueName: \"kubernetes.io/projected/c44bad23-f916-4198-abde-b8caa6a0a86a-kube-api-access-cxpld\") pod \"c44bad23-f916-4198-abde-b8caa6a0a86a\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") "
Jul 16 23:13:05.062761 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.062630    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bad23-f916-4198-abde-b8caa6a0a86a-openshift-service-ca\") pod \"c44bad23-f916-4198-abde-b8caa6a0a86a\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") "
Jul 16 23:13:05.062761 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.062685    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls\") pod \"c44bad23-f916-4198-abde-b8caa6a0a86a\" (UID: \"c44bad23-f916-4198-abde-b8caa6a0a86a\") "
Jul 16 23:13:05.063024 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.062999    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44bad23-f916-4198-abde-b8caa6a0a86a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c44bad23-f916-4198-abde-b8caa6a0a86a" (UID: "c44bad23-f916-4198-abde-b8caa6a0a86a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:13:05.064772 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.064752    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c44bad23-f916-4198-abde-b8caa6a0a86a-kube-api-access-cxpld" (OuterVolumeSpecName: "kube-api-access-cxpld") pod "c44bad23-f916-4198-abde-b8caa6a0a86a" (UID: "c44bad23-f916-4198-abde-b8caa6a0a86a"). InnerVolumeSpecName "kube-api-access-cxpld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:13:05.064836 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.064777    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c44bad23-f916-4198-abde-b8caa6a0a86a" (UID: "c44bad23-f916-4198-abde-b8caa6a0a86a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:13:05.129550 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.129519    2556 generic.go:356] "Generic (PLEG): container finished" podID="c44bad23-f916-4198-abde-b8caa6a0a86a" containerID="e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe" exitCode=0
Jul 16 23:13:05.129687 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.129568    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm" event={"ID":"c44bad23-f916-4198-abde-b8caa6a0a86a","Type":"ContainerDied","Data":"e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe"}
Jul 16 23:13:05.129687 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.129584    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"
Jul 16 23:13:05.129687 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.129614    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm" event={"ID":"c44bad23-f916-4198-abde-b8caa6a0a86a","Type":"ContainerDied","Data":"b1418a4f061c35f893f287129aa8867ee6ab1f0368286805ce8691fc0062f8e3"}
Jul 16 23:13:05.129687 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.129639    2556 scope.go:117] "RemoveContainer" containerID="e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe"
Jul 16 23:13:05.139478 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.139456    2556 scope.go:117] "RemoveContainer" containerID="e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe"
Jul 16 23:13:05.139743 ip-10-0-139-105 kubenswrapper[2556]: E0716 23:13:05.139723    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe\": container with ID starting with e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe not found: ID does not exist" containerID="e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe"
Jul 16 23:13:05.139807 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.139758    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe"} err="failed to get container status \"e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe\": rpc error: code = NotFound desc = could not find container \"e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe\": container with ID starting with e2bee4ab141ff7399268103d7815c968dd27725da28de42d75a6e91771f6ccbe not found: ID does not exist"
Jul 16 23:13:05.154268 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.154241    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"]
Jul 16 23:13:05.159807 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.159779    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-61f04e-6f86cc8b46-vnpkm"]
Jul 16 23:13:05.164015 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.163998    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c44bad23-f916-4198-abde-b8caa6a0a86a-maas-api-tls\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:13:05.164094 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.164017    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxpld\" (UniqueName: \"kubernetes.io/projected/c44bad23-f916-4198-abde-b8caa6a0a86a-kube-api-access-cxpld\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:13:05.164094 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.164027    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c44bad23-f916-4198-abde-b8caa6a0a86a-openshift-service-ca\") on node \"ip-10-0-139-105.ec2.internal\" DevicePath \"\""
Jul 16 23:13:05.812367 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:05.812338    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c44bad23-f916-4198-abde-b8caa6a0a86a" path="/var/lib/kubelet/pods/c44bad23-f916-4198-abde-b8caa6a0a86a/volumes"
Jul 16 23:13:09.162899 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:09.162865    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:15.165141 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:15.165089    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:15.229264 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:15.229233    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:19.823249 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:19.823213    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:20.047538 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:20.047500    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:25.544679 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:25.544637    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8cf7r_0a04d53a-c4aa-45e1-a423-df0ba3948bab/manager/0.log"
Jul 16 23:13:25.784843 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:25.784812    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79cbb885f5-zp56f_c4283ce5-4ba4-4ce6-b531-31ba1610d50e/manager/0.log"
Jul 16 23:13:25.895057 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:25.894975    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nlf8k_80bc537d-de13-46fa-bc76-15f1f096157b/server/0.log"
Jul 16 23:13:26.014716 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:26.014684    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ng8d4_eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3/manager/0.log"
Jul 16 23:13:26.257431 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:26.257357    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8_f62ec0fd-06de-4573-9942-2cc77b530d9a/copy-manifests/0.log"
Jul 16 23:13:26.276185 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:26.276151    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8_f62ec0fd-06de-4573-9942-2cc77b530d9a/manager/0.log"
Jul 16 23:13:26.753417 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:26.753380    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7954f7d7ff-rj66x_9ea89920-e85b-489a-b481-b82c342f57af/maas-api/0.log"
Jul 16 23:13:29.497556 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:29.497523    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-sg4d9_aca7c23d-c1f0-40e5-8048-b3d8b62b9a02/manager/0.log"
Jul 16 23:13:29.611659 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:29.611629    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mz287_54dd2528-227f-42ed-86a4-a8555cdeec0e/manager/0.log"
Jul 16 23:13:29.714879 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:29.714854    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zprlg_a9721a53-7880-477d-bf57-aeef00b239d0/kuadrant-console-plugin/0.log"
Jul 16 23:13:29.834573 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:29.834547    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jt5wq_644e0ca1-409b-4252-bcee-f13517f84d13/registry-server/0.log"
Jul 16 23:13:29.954440 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:29.954393    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cdnbt_24ac699f-66cc-4bc2-a22f-1087168aa9e2/manager/0.log"
Jul 16 23:13:30.175906 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:30.175820    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9488z_2402b484-8810-402d-ab88-5276cc81922a/manager/0.log"
Jul 16 23:13:30.745370 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:30.745340    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b969c6b8c-fjvw6_41d05597-6cf9-448f-8870-cac62db55f62/kube-auth-proxy/0.log"
Jul 16 23:13:31.102993 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:31.102958    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-hd7z8_13985c55-246e-4408-a04b-e7b95d6e72f3/payload-pre-processing/0.log"
Jul 16 23:13:31.326663 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:31.326632    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f9cf987d-cs7ph_ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072/router/0.log"
Jul 16 23:13:31.759724 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:31.759670    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-tdq9s_5ccfa151-01ea-4f55-9e6b-43253067498d/main/0.log"
Jul 16 23:13:31.868689 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:31.868660    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-n7sgh_c25fde28-2e3d-4cf3-8a43-86f2c09a05dd/main/0.log"
Jul 16 23:13:31.976296 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:31.976266    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db9lztt_3facf248-f2fe-486e-b35d-96cedba03e59/main/0.log"
Jul 16 23:13:37.398555 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:37.398509    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:37.498714 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:37.498676    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:40.311969 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:40.311888    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hhz6k_c4d9b940-f340-4329-aea0-6252d6d331d0/global-pull-secret-syncer/0.log"
Jul 16 23:13:40.405059 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:40.405026    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log"
Jul 16 23:13:40.520262 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:40.520231    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-105.ec2.internal_a88942796a5b6b2c5f36a7e709e10120/haproxy/0.log"
Jul 16 23:13:45.445746 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:45.445707    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-sg4d9_aca7c23d-c1f0-40e5-8048-b3d8b62b9a02/manager/0.log"
Jul 16 23:13:45.467174 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:45.467147    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mz287_54dd2528-227f-42ed-86a4-a8555cdeec0e/manager/0.log"
Jul 16 23:13:45.492459 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:45.492428    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zprlg_a9721a53-7880-477d-bf57-aeef00b239d0/kuadrant-console-plugin/0.log"
Jul 16 23:13:45.560245 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:45.560216    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jt5wq_644e0ca1-409b-4252-bcee-f13517f84d13/registry-server/0.log"
Jul 16 23:13:45.789321 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:45.789292    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-cdnbt_24ac699f-66cc-4bc2-a22f-1087168aa9e2/manager/0.log"
Jul 16 23:13:45.922015 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:45.921985    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9488z_2402b484-8810-402d-ab88-5276cc81922a/manager/0.log"
Jul 16 23:13:47.956587 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:47.956485    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-kfn8p_0db55b6f-7977-4d46-9528-a8dcf74de6cd/cluster-monitoring-operator/0.log"
Jul 16 23:13:48.197937 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:48.197912    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dlnzt_4ecd7152-05d7-44fd-9228-09ae575a2f28/node-exporter/0.log"
Jul 16 23:13:48.219924 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:48.219843    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dlnzt_4ecd7152-05d7-44fd-9228-09ae575a2f28/kube-rbac-proxy/0.log"
Jul 16 23:13:48.239162 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:48.239138    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dlnzt_4ecd7152-05d7-44fd-9228-09ae575a2f28/init-textfile/0.log"
Jul 16 23:13:50.071853 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.071821    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-m8xj9_b3eec9f2-9166-4cdd-8a4c-01b227fddc76/networking-console-plugin/0.log"
Jul 16 23:13:50.664181 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.664148    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/2.log"
Jul 16 23:13:50.672390 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.672366    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4b8xc_f36db1a6-eff5-45da-84a2-ca5d0572ac16/console-operator/3.log"
Jul 16 23:13:50.784311 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.784275    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"]
Jul 16 23:13:50.784753 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.784730    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79547d0-f91b-4d14-8f15-1805408088b9" containerName="maas-api"
Jul 16 23:13:50.784888 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.784851    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79547d0-f91b-4d14-8f15-1805408088b9" containerName="maas-api"
Jul 16 23:13:50.784963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.784900    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c44bad23-f916-4198-abde-b8caa6a0a86a" containerName="maas-api"
Jul 16 23:13:50.784963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.784911    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c44bad23-f916-4198-abde-b8caa6a0a86a" containerName="maas-api"
Jul 16 23:13:50.784963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.784929    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51feb54-74d8-4144-86ba-322eb524f812" containerName="test-curl-7525-140274772277552"
Jul 16 23:13:50.784963 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.784939    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51feb54-74d8-4144-86ba-322eb524f812" containerName="test-curl-7525-140274772277552"
Jul 16 23:13:50.785189 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.785062    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51feb54-74d8-4144-86ba-322eb524f812" containerName="test-curl-7525-140274772277552"
Jul 16 23:13:50.785189 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.785080    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c44bad23-f916-4198-abde-b8caa6a0a86a" containerName="maas-api"
Jul 16 23:13:50.785189 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.785094    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79547d0-f91b-4d14-8f15-1805408088b9" containerName="maas-api"
Jul 16 23:13:50.789625 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.789605    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.791967 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.791947    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvjp6\"/\"kube-root-ca.crt\""
Jul 16 23:13:50.791967 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.791959    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wvjp6\"/\"default-dockercfg-sgxhw\""
Jul 16 23:13:50.792172 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.791947    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wvjp6\"/\"openshift-service-ca.crt\""
Jul 16 23:13:50.795640 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.795615    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"]
Jul 16 23:13:50.876158 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.876081    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-sys\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.876158 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.876161    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-lib-modules\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.876368 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.876181    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/54ed0cf5-5979-4ae2-8220-3cdcc028246a-kube-api-access-wl9wh\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.876368 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.876204    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-podres\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.876368 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.876291    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-proc\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977406 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977320    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-proc\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977564 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977410    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-sys\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977564 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977439    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-lib-modules\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977564 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977458    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/54ed0cf5-5979-4ae2-8220-3cdcc028246a-kube-api-access-wl9wh\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977564 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977473    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-proc\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977564 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977473    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-sys\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977564 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977482    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-podres\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977777 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977578    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-podres\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.977777 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.977583    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54ed0cf5-5979-4ae2-8220-3cdcc028246a-lib-modules\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:50.985168 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:50.985136    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/54ed0cf5-5979-4ae2-8220-3cdcc028246a-kube-api-access-wl9wh\") pod \"perf-node-gather-daemonset-7bzv8\" (UID: \"54ed0cf5-5979-4ae2-8220-3cdcc028246a\") " pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:51.099980 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:51.099946    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:51.242453 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:51.242426    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"]
Jul 16 23:13:51.244552 ip-10-0-139-105 kubenswrapper[2556]: W0716 23:13:51.244526    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod54ed0cf5_5979_4ae2_8220_3cdcc028246a.slice/crio-6c7b17c381e476242e9be8e4ae9c68e61cc1aa65ea68202d5561603ccaa50e07 WatchSource:0}: Error finding container 6c7b17c381e476242e9be8e4ae9c68e61cc1aa65ea68202d5561603ccaa50e07: Status 404 returned error can't find the container with id 6c7b17c381e476242e9be8e4ae9c68e61cc1aa65ea68202d5561603ccaa50e07
Jul 16 23:13:51.380873 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:51.380826    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8" event={"ID":"54ed0cf5-5979-4ae2-8220-3cdcc028246a","Type":"ContainerStarted","Data":"3f6c057605e954ca4ed297aca69f901d7db56a9cf4a22c36a8e6b89696cdd0a1"}
Jul 16 23:13:51.381046 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:51.380915    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8" event={"ID":"54ed0cf5-5979-4ae2-8220-3cdcc028246a","Type":"ContainerStarted","Data":"6c7b17c381e476242e9be8e4ae9c68e61cc1aa65ea68202d5561603ccaa50e07"}
Jul 16 23:13:51.381046 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:51.381025    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:51.408373 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:51.408321    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8" podStartSLOduration=1.408307192 podStartE2EDuration="1.408307192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 23:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:13:51.406445683 +0000 UTC m=+3424.079704716" watchObservedRunningTime="2026-07-16 23:13:51.408307192 +0000 UTC m=+3424.081566199"
Jul 16 23:13:51.914310 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:51.914267    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-4qn8c_89bd5ddb-b578-43b1-9bc2-1610f4d858c0/volume-data-source-validator/0.log"
Jul 16 23:13:52.167428 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:52.167334    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:52.268018 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:52.267986    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjq4d_97d954b6-fe98-486b-a51e-a3020d766739/konnectivity-agent/0.log" limit=1024
Jul 16 23:13:52.940934 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:52.940903    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zxmt_a704ee01-ae37-4011-bb07-0a1fd7ca87b9/dns/0.log"
Jul 16 23:13:52.972835 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:52.972805    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zxmt_a704ee01-ae37-4011-bb07-0a1fd7ca87b9/kube-rbac-proxy/0.log"
Jul 16 23:13:53.120173 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:53.120139    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7nft5_fdc6c432-8858-4d49-b6ba-5c44ac2ed8d3/dns-node-resolver/0.log"
Jul 16 23:13:53.767529 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:53.767496    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8gvtz_4e4ac9ef-39d7-4fe0-b11a-2346d6e6a7e7/node-ca/0.log"
Jul 16 23:13:54.894210 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:54.894184    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b969c6b8c-fjvw6_41d05597-6cf9-448f-8870-cac62db55f62/kube-auth-proxy/0.log"
Jul 16 23:13:55.037639 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:55.037604    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-hd7z8_13985c55-246e-4408-a04b-e7b95d6e72f3/payload-pre-processing/0.log"
Jul 16 23:13:55.087996 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:55.087969    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f9cf987d-cs7ph_ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072/router/0.log"
Jul 16 23:13:55.791670 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:55.791640    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-49m5t_13f7c116-08cb-4fe3-9c54-ab12fb15b424/serve-healthcheck-canary/0.log"
Jul 16 23:13:56.304920 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:56.304886    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-sf45x_75f6164d-4c29-4111-bdaf-961f00f83f12/insights-operator/0.log"
Jul 16 23:13:56.308198 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:56.308173    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-sf45x_75f6164d-4c29-4111-bdaf-961f00f83f12/insights-operator/1.log"
Jul 16 23:13:56.328638 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:56.328607    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4qs4_be6d1ea1-7566-4a06-bca8-47d0a98525de/kube-rbac-proxy/0.log"
Jul 16 23:13:56.348519 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:56.348497    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4qs4_be6d1ea1-7566-4a06-bca8-47d0a98525de/exporter/0.log"
Jul 16 23:13:56.368812 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:56.368787    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4qs4_be6d1ea1-7566-4a06-bca8-47d0a98525de/extractor/0.log"
Jul 16 23:13:57.397161 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:57.397104    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wvjp6/perf-node-gather-daemonset-7bzv8"
Jul 16 23:13:58.577301 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:58.577268    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8cf7r_0a04d53a-c4aa-45e1-a423-df0ba3948bab/manager/0.log"
Jul 16 23:13:58.894726 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:58.894627    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79cbb885f5-zp56f_c4283ce5-4ba4-4ce6-b531-31ba1610d50e/manager/0.log"
Jul 16 23:13:58.916474 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:58.916440    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nlf8k_80bc537d-de13-46fa-bc76-15f1f096157b/server/0.log"
Jul 16 23:13:59.021607 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:59.021567    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ng8d4_eae71d11-3fbc-4b4c-89c2-a7fb52b46cb3/manager/0.log"
Jul 16 23:13:59.209773 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:59.209684    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8_f62ec0fd-06de-4573-9942-2cc77b530d9a/manager/0.log"
Jul 16 23:13:59.226906 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:13:59.226877    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cb9b8d7c8-dfqs8_f62ec0fd-06de-4573-9942-2cc77b530d9a/copy-manifests/0.log"
Jul 16 23:14:01.346253 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:01.346225    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b969c6b8c-fjvw6_41d05597-6cf9-448f-8870-cac62db55f62/kube-auth-proxy/0.log"
Jul 16 23:14:01.445671 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:01.445639    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-hd7z8_13985c55-246e-4408-a04b-e7b95d6e72f3/payload-pre-processing/0.log"
Jul 16 23:14:01.478016 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:01.477987    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f9cf987d-cs7ph_ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072/router/0.log"
Jul 16 23:14:02.819573 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:02.819542    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b969c6b8c-fjvw6_41d05597-6cf9-448f-8870-cac62db55f62/kube-auth-proxy/0.log"
Jul 16 23:14:02.933362 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:02.933329    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-hd7z8_13985c55-246e-4408-a04b-e7b95d6e72f3/payload-pre-processing/0.log"
Jul 16 23:14:02.968335 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:02.968300    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f9cf987d-cs7ph_ec6e9d8e-e10f-4a60-b5d6-1c449b2a5072/router/0.log"
Jul 16 23:14:05.718420 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:05.718391    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-6g5zn_1b785b66-f8f6-4280-b573-cee4f2694443/migrator/0.log"
Jul 16 23:14:05.738071 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:05.738047    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-6g5zn_1b785b66-f8f6-4280-b573-cee4f2694443/graceful-termination/0.log"
Jul 16 23:14:06.086323 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:06.086279    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-7c55f_4a08215e-447d-4500-8136-3a196037bc48/kube-storage-version-migrator-operator/1.log"
Jul 16 23:14:06.087991 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:06.087965    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-7c55f_4a08215e-447d-4500-8136-3a196037bc48/kube-storage-version-migrator-operator/0.log"
Jul 16 23:14:07.070635 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.070609    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cptpd_4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584/kube-multus-additional-cni-plugins/0.log"
Jul 16 23:14:07.091000 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.090972    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cptpd_4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584/egress-router-binary-copy/0.log"
Jul 16 23:14:07.110771 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.110741    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cptpd_4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584/cni-plugins/0.log"
Jul 16 23:14:07.130184 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.130162    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cptpd_4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584/bond-cni-plugin/0.log"
Jul 16 23:14:07.150270 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.150250    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cptpd_4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584/routeoverride-cni/0.log"
Jul 16 23:14:07.170155 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.170132    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cptpd_4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584/whereabouts-cni-bincopy/0.log"
Jul 16 23:14:07.189474 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.189451    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cptpd_4f94f52e-7e1a-4d9b-ad52-84bc9e5bd584/whereabouts-cni/0.log"
Jul 16 23:14:07.545432 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.545402    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dbspd_1226feab-5f42-4977-af53-d9d2957bd99e/kube-multus/0.log"
Jul 16 23:14:07.702679 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.702650    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fd92h_b45c42e2-acce-454f-8aa5-59e0b573e914/network-metrics-daemon/0.log"
Jul 16 23:14:07.722670 ip-10-0-139-105 kubenswrapper[2556]: I0716 23:14:07.722639    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fd92h_b45c42e2-acce-454f-8aa5-59e0b573e914/kube-rbac-proxy/0.log"