Jul 16 21:46:45.601530 ip-10-0-128-173 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 21:46:45.601543 ip-10-0-128-173 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 21:46:45.601551 ip-10-0-128-173 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 21:46:45.601803 ip-10-0-128-173 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 21:46:55.685349 ip-10-0-128-173 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 21:46:55.685371 ip-10-0-128-173 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2ffb88898c70442c9a8a3e227805ea12 --
Jul 16 21:48:53.490174 ip-10-0-128-173 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 21:48:53.903292 ip-10-0-128-173 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:48:53.903292 ip-10-0-128-173 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 21:48:53.903292 ip-10-0-128-173 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:48:53.903292 ip-10-0-128-173 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 21:48:53.903292 ip-10-0-128-173 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:48:53.904113 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.904009    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 21:48:53.908729 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908714    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:48:53.908729 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908729    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908732    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908736    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908738    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908743    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908746    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908749    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908752    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908755    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908757    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908760    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908762    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908765    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908769    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908772    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908775    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908778    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908781    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908788    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:48:53.908791 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908791    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908794    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908796    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908799    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908802    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908805    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908807    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908810    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908813    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908815    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908818    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908820    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908823    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908825    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908828    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908830    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908832    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908835    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908837    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908839    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:48:53.909297 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908842    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908845    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908848    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908850    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908853    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908855    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908858    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908860    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908864    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908868    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908871    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908874    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908877    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908879    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908882    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908885    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908887    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908890    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908892    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908895    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:48:53.909778 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908897    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908899    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908902    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908904    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908918    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908921    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908924    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908928    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908930    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908933    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908935    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908938    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908941    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908944    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908946    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908949    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908951    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908954    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908956    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908958    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:48:53.910274 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908961    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908963    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908966    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908968    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908970    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908973    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908976    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908978    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908981    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908983    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908986    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908988    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908991    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908993    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908996    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.908998    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909001    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909003    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909006    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909753    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:48:53.910856 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909760    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909764    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909767    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909770    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909774    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909778    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909781    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909784    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909787    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909790    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909792    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909795    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909797    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909800    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909802    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909805    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909807    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909810    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909813    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909815    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:48:53.911355 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909818    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909820    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909823    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909825    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909828    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909830    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909833    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909835    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909838    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909840    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909842    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909845    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909849    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909852    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909855    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909858    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909861    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909869    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909873    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:48:53.911883 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909875    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909878    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909880    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909883    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909885    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909887    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909892    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909895    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909898    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909900    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909903    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909922    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909925    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909928    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909931    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909934    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909936    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909939    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909941    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:48:53.912382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909944    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909946    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909949    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909951    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909954    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909956    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909959    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909961    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909963    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909966    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909969    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909973    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909983    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909986    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909988    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909991    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909994    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909996    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.909998    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910001    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:48:53.912863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910004    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910008    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910011    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910014    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910017    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910020    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910023    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910026    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910028    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910031    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910034    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910036    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910039    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910042    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910044    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910047    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910049    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910052    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910054    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910057    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:48:53.913363 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910427    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910431    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910434    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910437    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910440    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910443    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910446    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910449    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910451    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910454    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910459    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910461    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910464    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910466    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910469    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910471    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910474    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910477    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910479    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:48:53.913842 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910482    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910484    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910487    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910489    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910492    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910494    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910497    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910499    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910501    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910504    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910507    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910509    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910512    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910514    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910516    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910518    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910521    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910523    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910526    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910529    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:48:53.914322 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910531    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910534    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910536    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910540    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910543    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910546    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910548    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910553    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910556    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910559    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910562    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910565    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910567    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910570    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910572    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910575    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910578    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910580    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910583    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:48:53.914831 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910585    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910588    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910590    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910594    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910597    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910600    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910602    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910605    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910608    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910611    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910613    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910616    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910620    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910622    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910624    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910627    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910632    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910634    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910637    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:48:53.915310 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910639    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910642    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910645    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910648    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910652    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910655    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910658    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910660    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910662    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910665    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910668    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910671    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910673    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910675    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910678    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910681    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910683    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910686    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910688    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910691    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:48:53.915776 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910693    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.910696    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910771    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910782    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910792    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910798    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910808    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910814    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910820    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910826    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910829    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910832    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910836    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910839    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910842    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910845    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910848    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910851    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910854    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910857    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910862    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910864    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910867    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910870    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 21:48:53.916289 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910873    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910877    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910880    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910883    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910886    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910889    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910891    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910894    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910897    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910900    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910904    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910921    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910924    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910927    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910930    2564 flags.go:64] FLAG: --enable-server="true"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910933    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910937    2564 flags.go:64] FLAG: --event-burst="100"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910942    2564 flags.go:64] FLAG: --event-qps="50"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910945    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910948    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910951    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910957    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910960    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910963    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910965    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 16 21:48:53.916872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910968    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910971    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910974    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910977    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910980    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910982    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910985    2564 flags.go:64] FLAG: --feature-gates=""
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910989    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910991    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910994    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.910997    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911000    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911003    2564 flags.go:64] FLAG: --help="false"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911006    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-128-173.ec2.internal"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911009    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911011    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911014    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911017    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911020    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911023    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911025    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911028    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911031    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 21:48:53.917541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911034    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911039    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911041    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911044    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911047    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911051    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911053    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911056    2564 flags.go:64] FLAG: --lock-file=""
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911059    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911061    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911064    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911070    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911072    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911075    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911078    2564 flags.go:64] FLAG: --logging-format="text"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911080    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911084    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911087    2564 flags.go:64] FLAG: --manifest-url=""
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911090    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911094    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911097    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911101    2564 flags.go:64] FLAG: --max-pods="110"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911103    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911106    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911109    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 21:48:53.918133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911111    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911114    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911117    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911120    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911127    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911129    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911132    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911135    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911139    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911145    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911147    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911150    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911154    2564 flags.go:64] FLAG: --port="10250"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911157    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911160    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d1d0a464d10d3ca2"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911163    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911166    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911169    2564 flags.go:64] FLAG: --register-node="true"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911171    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911175    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911178    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911180    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911183    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911186    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911189    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 21:48:53.918737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911192    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911195    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911197    2564 flags.go:64] FLAG: --runonce="false"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911200    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911203    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911205    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911208    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911211    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911213    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911216    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911219    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911221    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911224    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911226    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911229    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911232    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911236    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911239    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911244    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911248    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911251    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911255    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911258    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911261    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911263    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 21:48:53.919363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911266    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911269    2564 flags.go:64] FLAG: --v="2"
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911272    2564 flags.go:64] FLAG: --version="false"
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911276    2564 flags.go:64] FLAG: --vmodule=""
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911279    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911282    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911405    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911408    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911411    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911413    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911416    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911419    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911422    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911424    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911427    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911429    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911432    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911434    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911437    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911440    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911442    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:48:53.919994 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911445    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911447    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911450    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911453    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911456    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911459    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911462    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911465    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911467    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911469    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911472    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911474    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911477    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911479    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911482    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911484    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911486    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911489    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911491    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911493    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:48:53.920502 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911496    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911498    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911501    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911503    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911505    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911509    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911513    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911515    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911518    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911521    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911523    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911526    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911529    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911532    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911535    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911537    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911541    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911545    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911548    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911550    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:48:53.921114 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911553    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911555    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911558    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911561    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911563    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911565    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911568    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911570    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911573    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911575    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911577    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911580    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911582    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911585    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911587    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911590    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911592    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911594    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911597    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911601    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:48:53.921604 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911604    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911606    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911610    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911613    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911616    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911619    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911622    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911624    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911627    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911631    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911634    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911636    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911638    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911641    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911643    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911646    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911648    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911650    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911653    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:48:53.922105 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911656    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911658    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911661    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911663    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.911666    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.911674    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.918624    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.918638    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.918656    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.918662    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918726    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918731    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918734    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918737    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918740    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:48:53.922560 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918748    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918751    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918754    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918757    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918759    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918762    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918764    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918773    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918776    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918778    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918781    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918783    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918786    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918788    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918791    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918793    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918796    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918799    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918801    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918807    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:48:53.922946 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918810    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918812    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918815    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918817    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918820    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918823    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918825    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918828    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918830    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918832    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918835    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918837    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918840    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918842    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918845    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918847    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918849    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918852    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918854    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918857    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:48:53.923443 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918864    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918867    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918869    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918872    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918874    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918876    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918879    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918882    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918884    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918887    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918889    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918892    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918895    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918897    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918900    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918902    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918905    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918923    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918928    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:48:53.923950 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918932    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918935    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918938    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918941    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918944    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918946    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918949    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918951    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918954    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918956    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918959    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918961    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918964    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918966    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918975    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918977    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918981    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918984    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918987    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918989    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:48:53.924451 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918992    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918995    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.918997    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919000    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919002    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919005    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919007    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919009    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919012    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919015    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919018    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919022    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919024    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919027    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:48:53.925041 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919029    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.919034    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919184    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919189    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919191    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919194    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919197    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919199    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919202    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919204    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919207    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919209    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919211    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919220    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:48:53.925382 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919223    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919225    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919228    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919230    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919232    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919235    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919237    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919239    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919242    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919244    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919246    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919249    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919251    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919254    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919256    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919259    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919261    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919263    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919266    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919268    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:48:53.925728 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919270    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919273    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919275    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919278    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919280    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919282    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919285    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919287    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919289    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919292    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919294    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919296    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919304    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919306    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919309    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919311    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919313    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919316    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919318    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919321    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:48:53.926230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919323    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919325    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919329    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919332    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919334    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919337    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919340    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919343    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919345    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919348    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919350    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919352    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919355    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919357    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919360    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919362    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919364    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919367    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919369    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:48:53.926878 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919372    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919374    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919377    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919379    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919382    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919385    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919392    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919395    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919398    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919400    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919403    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919405    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919407    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919410    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919412    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919414    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919417    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919419    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919422    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919424    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:48:53.927512 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919427    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919429    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919431    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919435    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919439    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919442    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919445    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:53.919447    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.919452    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.919562    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.922413    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.923825    2564 server.go:1019] "Starting client certificate rotation"
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.923939    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:48:53.928306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.923975    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:48:53.928796 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.928761    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:48:53.929957 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.929933    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:48:53.941049 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.941028    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 16 21:48:53.942982 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.942968    2564 log.go:25] "Validated CRI v1 image API"
Jul 16 21:48:53.944692 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.944672    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 21:48:53.951421 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.951402    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ad40488b-bcf8-41b0-97bb-65493ba1e86b:/dev/nvme0n1p4 cfcec2e5-c7ba-4e8a-8153-3c133e72aa6f:/dev/nvme0n1p3]
Jul 16 21:48:53.951489 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.951421    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 21:48:53.958285 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.958172    2564 manager.go:216] Machine: {Timestamp:2026-07-16 21:48:53.956796211 +0000 UTC m=+0.346069989 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101310 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26a71a757785b47266d66e7f1fb1c9 SystemUUID:ec26a71a-7577-85b4-7266-d66e7f1fb1c9 BootID:2ffb8889-8c70-442c-9a8a-3e227805ea12 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f6:c1:a4:f8:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f6:c1:a4:f8:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:87:3b:a7:a7:a2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 21:48:53.958285 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.958278    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 21:48:53.958402 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.958355    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 21:48:53.961183 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.961143    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 21:48:53.961297 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.961275    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:48:53.961372 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.961188    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 21:48:53.961874 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.961863    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 21:48:53.961922 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.961876    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 21:48:53.961922 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.961884    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:48:53.963345 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.963334    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:48:53.964580 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.964567    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 21:48:53.967806 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.967794    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:48:53.967958 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.967948    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 21:48:53.967991 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.967981    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 21:48:53.968033 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.968001    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 21:48:53.968033 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.968025    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 21:48:53.968104 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.968050    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 16 21:48:53.968153 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.968131    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 21:48:53.968696 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.968678    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:48:53.968696 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.968700    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:48:53.972232 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.972214    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 21:48:53.974095 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.974081    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 21:48:53.974153 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.974112    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 21:48:53.975164 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975152    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 21:48:53.975211 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975168    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 21:48:53.975211 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975174    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 21:48:53.975211 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975186    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 21:48:53.975211 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975199    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 21:48:53.975211 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975208    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 21:48:53.975345 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975217    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 21:48:53.975345 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975222    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 21:48:53.975345 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975229    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 21:48:53.975345 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975235    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 21:48:53.975345 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975243    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 21:48:53.975345 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.975248    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 21:48:53.976088 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.976078    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 21:48:53.976088 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.976087    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 21:48:53.979393 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.979376    2564 server.go:1268] "Started kubelet"
Jul 16 21:48:53.979508 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.979466    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 21:48:53.979593 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.979511    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 21:48:53.979593 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.979581    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 21:48:53.980279 ip-10-0-128-173 systemd[1]: Started Kubernetes Kubelet.
Jul 16 21:48:53.981083 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.981065    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 21:48:53.981994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.981979    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 21:48:53.982991 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.982969    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 21:48:53.983062 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:53.982992    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 21:48:53.983062 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:53.983030    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 21:48:53.984884 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:53.984867    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 21:48:53.987720 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.987698    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 21:48:53.987720 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.987715    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 21:48:53.987865 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:53.987802    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:53.988431 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988417    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 21:48:53.988554 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988420    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 21:48:53.988554 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988555    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 21:48:53.988717 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988697    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 21:48:53.988717 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988706    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 21:48:53.988717 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988712    2564 factory.go:153] Registering CRI-O factory
Jul 16 21:48:53.988863 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988725    2564 factory.go:223] Registration of the crio container factory successfully
Jul 16 21:48:53.988863 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988771    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 21:48:53.988863 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988780    2564 factory.go:55] Registering systemd factory
Jul 16 21:48:53.988863 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988786    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 16 21:48:53.988863 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988801    2564 factory.go:103] Registering Raw factory
Jul 16 21:48:53.988863 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.988815    2564 manager.go:1196] Started watching for new ooms in manager
Jul 16 21:48:53.989381 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:53.989273    2564 manager.go:349] Starting recovery of all containers
Jul 16 21:48:53.992254 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:53.988731    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c2e37e4c6720d5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-16 21:48:53.979349205 +0000 UTC m=+0.368622982,LastTimestamp:2026-07-16 21:48:53.979349205 +0000 UTC m=+0.368622982,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 16 21:48:53.995255 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:53.995219    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 21:48:53.995566 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:53.995538    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 21:48:54.000529 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.000511    2564 manager.go:354] Recovery completed
Jul 16 21:48:54.004566 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.004555    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:48:54.006269 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.006246    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:48:54.006322 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.006280    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:48:54.006322 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.006291    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:48:54.006816 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.006802    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 21:48:54.006816 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.006808    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:48:54.006816 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.006814    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 21:48:54.006960 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.006829    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:48:54.007488 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.007470    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:48:54.007559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.007498    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:48:54.007559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.007507    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:48:54.008048 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.008035    2564 policy_none.go:49] "None policy: Start"
Jul 16 21:48:54.008102 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.008055    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 21:48:54.008102 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.008069    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 21:48:54.008998 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.008895    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c2e37e4e01e1af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-16 21:48:54.006268335 +0000 UTC m=+0.395542112,LastTimestamp:2026-07-16 21:48:54.006268335 +0000 UTC m=+0.395542112,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 16 21:48:54.009111 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.009095    2564 policy_none.go:47] "Start"
Jul 16 21:48:54.018185 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.018033    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c2e37e4e0224d3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-16 21:48:54.006285523 +0000 UTC m=+0.395559303,LastTimestamp:2026-07-16 21:48:54.006285523 +0000 UTC m=+0.395559303,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 16 21:48:54.021899 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.021842    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c2e37e4e0247b5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-16 21:48:54.006294453 +0000 UTC m=+0.395568231,LastTimestamp:2026-07-16 21:48:54.006294453 +0000 UTC m=+0.395568231,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 16 21:48:54.027562 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.027491    2564 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-173.ec2.internal.18c2e37e4e01e1af\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c2e37e4e01e1af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-16 21:48:54.006268335 +0000 UTC m=+0.395542112,LastTimestamp:2026-07-16 21:48:54.007485436 +0000 UTC m=+0.396759215,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 16 21:48:54.034597 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.034540    2564 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-173.ec2.internal.18c2e37e4e0224d3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c2e37e4e0224d3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-16 21:48:54.006285523 +0000 UTC m=+0.395559303,LastTimestamp:2026-07-16 21:48:54.007502721 +0000 UTC m=+0.396776501,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 16 21:48:54.038961 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.038942    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gx5l7"
Jul 16 21:48:54.045449 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.045394    2564 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-173.ec2.internal.18c2e37e4e0247b5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c2e37e4e0247b5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-16 21:48:54.006294453 +0000 UTC m=+0.395568231,LastTimestamp:2026-07-16 21:48:54.007511332 +0000 UTC m=+0.396785111,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 16 21:48:54.048529 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.048515    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gx5l7"
Jul 16 21:48:54.049460 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.049446    2564 manager.go:337] "Starting Device Plugin manager"
Jul 16 21:48:54.049536 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.049480    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 21:48:54.049536 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.049492    2564 server.go:87] "Starting device plugin registration server"
Jul 16 21:48:54.049721 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.049707    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 21:48:54.049776 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.049724    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 21:48:54.049826 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.049819    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 21:48:54.049931 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.049903    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 21:48:54.049931 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.049931    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 21:48:54.050880 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.050861    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 21:48:54.050974 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.050933    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.117150 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.117119    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 21:48:54.118582 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.118564    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 21:48:54.118657 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.118607    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 21:48:54.118657 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.118627    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 21:48:54.118760 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.118658    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 21:48:54.122282 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.122265    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:48:54.149854 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.149819    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:48:54.150709 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.150693    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:48:54.150801 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.150721    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:48:54.150801 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.150732    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:48:54.150801 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.150757    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.168269 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.168222    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.168269 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.168242    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-173.ec2.internal\": node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.211139 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.211117    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.218797 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.218775    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"]
Jul 16 21:48:54.218884 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.218842    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:48:54.219686 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.219669    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:48:54.219757 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.219698    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:48:54.219757 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.219708    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:48:54.220938 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.220926    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:48:54.221045 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221021    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.221086 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221049    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:48:54.221625 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221611    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:48:54.221707 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221635    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:48:54.221707 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221645    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:48:54.221810 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221715    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:48:54.221810 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221750    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:48:54.221810 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.221765    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:48:54.222864 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.222849    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.222936 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.222879    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:48:54.223506 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.223490    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:48:54.223587 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.223517    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:48:54.223587 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.223527    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:48:54.248568 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.248547    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-173.ec2.internal\" not found" node="ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.252885 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.252870    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-173.ec2.internal\" not found" node="ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.291148 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.291121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bae453749cedffbd6b92c7f878ca7154-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"bae453749cedffbd6b92c7f878ca7154\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.291240 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.291154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bae453749cedffbd6b92c7f878ca7154-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"bae453749cedffbd6b92c7f878ca7154\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.291240 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.291179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9617a17fc683a92e3fbbd52c642b5794-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"9617a17fc683a92e3fbbd52c642b5794\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.311180 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.311156    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.391902 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.391882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9617a17fc683a92e3fbbd52c642b5794-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"9617a17fc683a92e3fbbd52c642b5794\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.392011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.391930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bae453749cedffbd6b92c7f878ca7154-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"bae453749cedffbd6b92c7f878ca7154\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.392011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.391958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bae453749cedffbd6b92c7f878ca7154-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"bae453749cedffbd6b92c7f878ca7154\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.392011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.391961    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9617a17fc683a92e3fbbd52c642b5794-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"9617a17fc683a92e3fbbd52c642b5794\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.392011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.391984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bae453749cedffbd6b92c7f878ca7154-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"bae453749cedffbd6b92c7f878ca7154\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.392172 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.392011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bae453749cedffbd6b92c7f878ca7154-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"bae453749cedffbd6b92c7f878ca7154\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.411431 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.411400    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.511516 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.511456    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.552670 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.552646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.555435 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.555419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.611551 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.611509    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.712055 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.712026    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.812659 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.812593    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.913378 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.913344    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 16 21:48:54.923240 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.923203    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 21:48:54.923397 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.923376    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 21:48:54.967175 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.967149    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:48:54.969314 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.969299    2564 apiserver.go:52] "Watching apiserver"
Jul 16 21:48:54.975330 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.975299    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:48:54.979122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.979099    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:48:54.981629 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.981612    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 21:48:54.982606 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.982585    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-tm8hd","kube-system/global-pull-secret-syncer-zkdf9","kube-system/konnectivity-agent-7vc4n","openshift-cluster-node-tuning-operator/tuned-4hwcf","openshift-multus/multus-additional-cni-plugins-czz97","openshift-multus/network-metrics-daemon-hzwzf","openshift-network-operator/iptables-alerter-6kcq4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs","openshift-dns/node-resolver-s8d4d","openshift-image-registry/node-ca-nwfwh","openshift-multus/multus-xc6wt","openshift-network-diagnostics/network-check-target-8ltd5"]
Jul 16 21:48:54.984056 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.984040    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:54.985085 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.985067    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:54.985185 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.985145    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:48:54.986287 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.986268    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:54.987386 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.987366    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:54.987852 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.987830    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 21:48:54.988352 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.988329    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:54.988542 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.988528    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.989661 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.989643    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:54.989744 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.989701    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:48:54.990883 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.990866    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:54.992410 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.992394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:54.993663 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.993646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:54.995044 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.994982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:54.995144 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6mfl\" (UniqueName: \"kubernetes.io/projected/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-kube-api-access-s6mfl\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:54.995144 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f9adec6-0471-441e-8c76-2dbdd18bfc18-dbus\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:54.995144 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995107    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995144 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995131    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovn-node-metrics-cert\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6rhm\" (UniqueName: \"kubernetes.io/projected/a215a163-d0cc-42a2-87cb-c50e41d39cfc-kube-api-access-w6rhm\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crbn\" (UniqueName: \"kubernetes.io/projected/9d31ef25-6751-41d8-9583-11a3455909e6-kube-api-access-9crbn\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995252    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-ovn\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995324    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovnkube-config\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovnkube-script-lib\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995379    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/234d30fa-4935-4b4e-9091-9dbc56829631-konnectivity-ca\") pod \"konnectivity-agent-7vc4n\" (UID: \"234d30fa-4935-4b4e-9091-9dbc56829631\") " pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-slash\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-run-netns\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995460    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-cni-bin\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995488    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-hosts-file\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-serviceca\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-systemd-units\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f9adec6-0471-441e-8c76-2dbdd18bfc18-kubelet-config\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995636    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-systemd\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995658    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-etc-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-env-overrides\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-host\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-var-lib-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995799    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-log-socket\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-tmp-dir\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995860    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-kubelet\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.995907 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-node-log\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.996741 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-cni-netd\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:54.996741 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.995999    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/234d30fa-4935-4b4e-9091-9dbc56829631-agent-certs\") pod \"konnectivity-agent-7vc4n\" (UID: \"234d30fa-4935-4b4e-9091-9dbc56829631\") " pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:54.996741 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.996030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbhjt\" (UniqueName: \"kubernetes.io/projected/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-kube-api-access-vbhjt\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:54.996741 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.996620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:54.997784 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:54.997763    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:54.997882 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:54.997825    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:48:55.006528 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006502    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ptkh8\""
Jul 16 21:48:55.006670 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006638    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 21:48:55.006815 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006684    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 21:48:55.006815 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006737    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 21:48:55.006815 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006776    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 21:48:55.006815 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006798    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 21:48:55.007151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006839    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:48:55.007151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006776    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 21:48:55.007151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006890    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 21:48:55.007151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006841    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 21:48:55.007151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.006840    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 21:48:55.007151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007014    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 21:48:55.007418 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007215    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 21:48:55.007418 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007238    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 21:48:55.007494 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007445    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 21:48:55.007584 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007566    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007775    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rqhrz\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007804    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sfk48\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007844    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6ppmw\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007852    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007777    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-872gw\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007859    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007905    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007976    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007964    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gt7pf\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.007852    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.008044    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.008063    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 21:48:55.008163 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.008145    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tfjd5\""
Jul 16 21:48:55.008783 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.008230    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 21:48:55.008783 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.008283    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 21:48:55.008783 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.008447    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-whkjc\""
Jul 16 21:48:55.009806 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.009786    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 21:48:55.009893 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.009786    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 21:48:55.018433 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.018415    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bcb4d\""
Jul 16 21:48:55.022006 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.021982    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:48:55.022383 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.022368    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 16 21:48:55.022541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.022524    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"]
Jul 16 21:48:55.037100 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.037073    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:48:55.037693 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.037673    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 21:48:55.052937 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.052886    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 21:43:54 +0000 UTC" deadline="2028-01-25 19:33:46.404234266 +0000 UTC"
Jul 16 21:48:55.052937 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.052936    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13389h44m51.351301916s"
Jul 16 21:48:55.089518 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.089471    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 21:48:55.096726 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096705    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-tuned\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.096824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cni-binary-copy\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.096824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qsc9\" (UniqueName: \"kubernetes.io/projected/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-kube-api-access-9qsc9\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.096824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096791    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-system-cni-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.096824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096816    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-kubelet\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-conf-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6jzv\" (UniqueName: \"kubernetes.io/projected/56254e7e-c6df-4831-a4da-c5f1b9e99736-kube-api-access-v6jzv\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-tmp-dir\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.096986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-kubelet\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-cni-netd\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097038    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-sys-fs\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097064    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-multus-certs\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-kubelet\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097087    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-systemd\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.097118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-cni-netd\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cnibin\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-device-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097189    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f9adec6-0471-441e-8c76-2dbdd18bfc18-dbus\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovn-node-metrics-cert\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6rhm\" (UniqueName: \"kubernetes.io/projected/a215a163-d0cc-42a2-87cb-c50e41d39cfc-kube-api-access-w6rhm\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097278    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f9adec6-0471-441e-8c76-2dbdd18bfc18-dbus\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097291    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-tmp-dir\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-var-lib-kubelet\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d56nl\" (UniqueName: \"kubernetes.io/projected/22d2b7e9-dfe9-4207-b112-d9ca06c42891-kube-api-access-d56nl\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097370    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-cni-bin\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097456    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-hostroot\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-hosts-file\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:55.097559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-systemd-units\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097564    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097571    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097587    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-hosts-file\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysconfig\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.097620    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-systemd-units\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f9adec6-0471-441e-8c76-2dbdd18bfc18-kubelet-config\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-systemd\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f9adec6-0471-441e-8c76-2dbdd18bfc18-kubelet-config\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.097706    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret podName:5f9adec6-0471-441e-8c76-2dbdd18bfc18 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:55.597679911 +0000 UTC m=+1.986953723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret") pod "global-pull-secret-syncer-zkdf9" (UID: "5f9adec6-0471-441e-8c76-2dbdd18bfc18") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097719    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-systemd\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-etc-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-env-overrides\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097801    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-etc-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097803    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-iptables-alerter-script\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.098353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097833    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-sys\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-os-release\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvtg7\" (UniqueName: \"kubernetes.io/projected/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-kube-api-access-tvtg7\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-host\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-host\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.097990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-log-socket\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-lib-modules\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-daemon-config\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098055    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-node-log\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-kubernetes\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-host\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098133    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-log-socket\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098159    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-registration-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-node-log\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-cni-multus\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/234d30fa-4935-4b4e-9091-9dbc56829631-agent-certs\") pod \"konnectivity-agent-7vc4n\" (UID: \"234d30fa-4935-4b4e-9091-9dbc56829631\") " pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbhjt\" (UniqueName: \"kubernetes.io/projected/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-kube-api-access-vbhjt\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:55.099063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6mfl\" (UniqueName: \"kubernetes.io/projected/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-kube-api-access-s6mfl\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysctl-conf\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-system-cni-dir\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-env-overrides\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098363    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098388    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-socket-dir-parent\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-etc-kubernetes\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098634    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098657    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-etc-selinux\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-cnibin\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098701    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56254e7e-c6df-4831-a4da-c5f1b9e99736-cni-binary-copy\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-run\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098755    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-run-ovn-kubernetes\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22d2b7e9-dfe9-4207-b112-d9ca06c42891-tmp\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.098825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.098971    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crbn\" (UniqueName: \"kubernetes.io/projected/9d31ef25-6751-41d8-9583-11a3455909e6-kube-api-access-9crbn\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:55.100034 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099075    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-ovn\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.099127    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:55.599104327 +0000 UTC m=+1.988378107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099150    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099180    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-run-ovn\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovnkube-config\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovnkube-script-lib\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/234d30fa-4935-4b4e-9091-9dbc56829631-konnectivity-ca\") pod \"konnectivity-agent-7vc4n\" (UID: \"234d30fa-4935-4b4e-9091-9dbc56829631\") " pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-slash\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-run-netns\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-cni-bin\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvmld\" (UniqueName: \"kubernetes.io/projected/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-kube-api-access-tvmld\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099407    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysctl-d\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099423    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-os-release\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099447    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-netns\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-serviceca\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:55.100565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-modprobe-d\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099517    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-socket-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099562    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-cni-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099578    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-k8s-cni-cncf-io\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-host-slash\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-var-lib-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099677    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-var-lib-openvswitch\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099706    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-slash\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099731    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-run-netns\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099754    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a215a163-d0cc-42a2-87cb-c50e41d39cfc-host-cni-bin\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovnkube-script-lib\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/234d30fa-4935-4b4e-9091-9dbc56829631-konnectivity-ca\") pod \"konnectivity-agent-7vc4n\" (UID: \"234d30fa-4935-4b4e-9091-9dbc56829631\") " pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.099926    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovnkube-config\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.100135    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-serviceca\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:55.101109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.101094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a215a163-d0cc-42a2-87cb-c50e41d39cfc-ovn-node-metrics-cert\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.101734 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.101185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/234d30fa-4935-4b4e-9091-9dbc56829631-agent-certs\") pod \"konnectivity-agent-7vc4n\" (UID: \"234d30fa-4935-4b4e-9091-9dbc56829631\") " pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:55.115829 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.115808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6rhm\" (UniqueName: \"kubernetes.io/projected/a215a163-d0cc-42a2-87cb-c50e41d39cfc-kube-api-access-w6rhm\") pod \"ovnkube-node-tm8hd\" (UID: \"a215a163-d0cc-42a2-87cb-c50e41d39cfc\") " pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.118077 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.118058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbhjt\" (UniqueName: \"kubernetes.io/projected/6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41-kube-api-access-vbhjt\") pod \"node-resolver-s8d4d\" (UID: \"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41\") " pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:55.120052 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.120026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crbn\" (UniqueName: \"kubernetes.io/projected/9d31ef25-6751-41d8-9583-11a3455909e6-kube-api-access-9crbn\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:55.121341 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.121322    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-62ct8"
Jul 16 21:48:55.128802 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.128761    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae453749cedffbd6b92c7f878ca7154.slice/crio-65f407c607e051f17df8b5b70fa6021ac1ffdf58c4dc1e5b99e0ba801f439c07 WatchSource:0}: Error finding container 65f407c607e051f17df8b5b70fa6021ac1ffdf58c4dc1e5b99e0ba801f439c07: Status 404 returned error can't find the container with id 65f407c607e051f17df8b5b70fa6021ac1ffdf58c4dc1e5b99e0ba801f439c07
Jul 16 21:48:55.129128 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.129097    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9617a17fc683a92e3fbbd52c642b5794.slice/crio-bf7c90e72b7ac024810136b59d1aa61c288be0509139e465c117b4aab2ebb404 WatchSource:0}: Error finding container bf7c90e72b7ac024810136b59d1aa61c288be0509139e465c117b4aab2ebb404: Status 404 returned error can't find the container with id bf7c90e72b7ac024810136b59d1aa61c288be0509139e465c117b4aab2ebb404
Jul 16 21:48:55.133976 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.133940    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:48:55.135702 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.135678    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6mfl\" (UniqueName: \"kubernetes.io/projected/e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e-kube-api-access-s6mfl\") pod \"node-ca-nwfwh\" (UID: \"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e\") " pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:55.138095 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.138078    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:48:55.138202 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.138185    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"]
Jul 16 21:48:55.145662 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.145645    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-62ct8"
Jul 16 21:48:55.200681 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-host-slash\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200688    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-tuned\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cni-binary-copy\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qsc9\" (UniqueName: \"kubernetes.io/projected/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-kube-api-access-9qsc9\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200745    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-system-cni-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-kubelet\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-conf-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200806    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-host-slash\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.200824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6jzv\" (UniqueName: \"kubernetes.io/projected/56254e7e-c6df-4831-a4da-c5f1b9e99736-kube-api-access-v6jzv\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-sys-fs\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200869    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-kubelet\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200873    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-system-cni-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-multus-certs\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-conf-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200970    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-multus-certs\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.200973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-sys-fs\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201001    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-systemd\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-systemd\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cnibin\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cnibin\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-device-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-device-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201168    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-var-lib-kubelet\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.201201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201177    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d56nl\" (UniqueName: \"kubernetes.io/projected/22d2b7e9-dfe9-4207-b112-d9ca06c42891-kube-api-access-d56nl\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-var-lib-kubelet\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201225    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-cni-bin\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-hostroot\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201317    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-cni-bin\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysconfig\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201368    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-hostroot\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201373    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cni-binary-copy\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysconfig\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201383    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-iptables-alerter-script\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-sys\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-os-release\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvtg7\" (UniqueName: \"kubernetes.io/projected/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-kube-api-access-tvtg7\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-lib-modules\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-daemon-config\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.202043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-kubernetes\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201627    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-sys\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201637    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-os-release\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201678    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-kubernetes\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-host\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201723    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-registration-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-cni-multus\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysctl-conf\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-lib-modules\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201723    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-host\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201822    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-registration-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-var-lib-cni-multus\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201880    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-system-cni-dir\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201926    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysctl-conf\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-iptables-alerter-script\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.202837 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-system-cni-dir\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.201974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-socket-dir-parent\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-etc-kubernetes\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-socket-dir-parent\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-etc-selinux\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-cnibin\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-etc-kubernetes\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202447    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56254e7e-c6df-4831-a4da-c5f1b9e99736-cni-binary-copy\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-run\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22d2b7e9-dfe9-4207-b112-d9ca06c42891-tmp\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-etc-selinux\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvmld\" (UniqueName: \"kubernetes.io/projected/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-kube-api-access-tvmld\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202647    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysctl-d\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-os-release\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202727    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-netns\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-modprobe-d\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202809    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-socket-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.203710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202888    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-daemon-config\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202957    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-cni-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.202988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-netns\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-sysctl-d\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203121    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-socket-dir\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-cnibin\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203202    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-modprobe-d\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56254e7e-c6df-4831-a4da-c5f1b9e99736-cni-binary-copy\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203257    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/22d2b7e9-dfe9-4207-b112-d9ca06c42891-run\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203257    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-os-release\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-multus-cni-dir\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-k8s-cni-cncf-io\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56254e7e-c6df-4831-a4da-c5f1b9e99736-host-run-k8s-cni-cncf-io\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.203698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.204550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.204257    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/22d2b7e9-dfe9-4207-b112-d9ca06c42891-etc-tuned\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.205303 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.205253    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22d2b7e9-dfe9-4207-b112-d9ca06c42891-tmp\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.222090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.222063    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:48:55.222090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.222084    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:48:55.222229 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.222099    2564 projected.go:196] Error preparing data for projected volume kube-api-access-tlr2l for pod openshift-network-diagnostics/network-check-target-8ltd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:55.222229 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.222156    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l podName:4cbd2da1-d5ea-41f9-a004-dd503966a636 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:55.722141363 +0000 UTC m=+2.111415150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tlr2l" (UniqueName: "kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l") pod "network-check-target-8ltd5" (UID: "4cbd2da1-d5ea-41f9-a004-dd503966a636") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:55.224139 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.224109    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvmld\" (UniqueName: \"kubernetes.io/projected/e51e6ee9-1ce5-4258-8ee5-e974a38a07ae-kube-api-access-tvmld\") pod \"iptables-alerter-6kcq4\" (UID: \"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae\") " pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.225122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.225102    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qsc9\" (UniqueName: \"kubernetes.io/projected/f9fd0b51-ccba-49b6-8ea2-26dd143d4916-kube-api-access-9qsc9\") pod \"aws-ebs-csi-driver-node-wf2vs\" (UID: \"f9fd0b51-ccba-49b6-8ea2-26dd143d4916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.230710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.230687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d56nl\" (UniqueName: \"kubernetes.io/projected/22d2b7e9-dfe9-4207-b112-d9ca06c42891-kube-api-access-d56nl\") pod \"tuned-4hwcf\" (UID: \"22d2b7e9-dfe9-4207-b112-d9ca06c42891\") " pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.232389 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.232367    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvtg7\" (UniqueName: \"kubernetes.io/projected/eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e-kube-api-access-tvtg7\") pod \"multus-additional-cni-plugins-czz97\" (UID: \"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e\") " pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.233095 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.233076    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6jzv\" (UniqueName: \"kubernetes.io/projected/56254e7e-c6df-4831-a4da-c5f1b9e99736-kube-api-access-v6jzv\") pod \"multus-xc6wt\" (UID: \"56254e7e-c6df-4831-a4da-c5f1b9e99736\") " pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.317825 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.317800    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:48:55.327005 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.326963    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234d30fa_4935_4b4e_9091_9dbc56829631.slice/crio-8649009008b3f60da730f8cb139cbb68dfacbe09b070936dadb31d399168795f WatchSource:0}: Error finding container 8649009008b3f60da730f8cb139cbb68dfacbe09b070936dadb31d399168795f: Status 404 returned error can't find the container with id 8649009008b3f60da730f8cb139cbb68dfacbe09b070936dadb31d399168795f
Jul 16 21:48:55.336065 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.336043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s8d4d"
Jul 16 21:48:55.343526 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.343504    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cdca38f_6c5c_4ab2_a93d_779ac9a7aa41.slice/crio-1de362d8611b30161a7d224b65edbdb4d0dba8c5736823dbc4eb9bd833a20e57 WatchSource:0}: Error finding container 1de362d8611b30161a7d224b65edbdb4d0dba8c5736823dbc4eb9bd833a20e57: Status 404 returned error can't find the container with id 1de362d8611b30161a7d224b65edbdb4d0dba8c5736823dbc4eb9bd833a20e57
Jul 16 21:48:55.357439 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.357421    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nwfwh"
Jul 16 21:48:55.364487 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.364456    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c0dc4b_aa01_4b81_a48d_4cd6584e3c4e.slice/crio-8a3728acb73d713c764d4f283ea647ed346e49705d71aec41f86d1e2a033b1a4 WatchSource:0}: Error finding container 8a3728acb73d713c764d4f283ea647ed346e49705d71aec41f86d1e2a033b1a4: Status 404 returned error can't find the container with id 8a3728acb73d713c764d4f283ea647ed346e49705d71aec41f86d1e2a033b1a4
Jul 16 21:48:55.366938 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.366844    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:48:55.373748 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.373730    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda215a163_d0cc_42a2_87cb_c50e41d39cfc.slice/crio-2d81ff558c15169b19b213afa552c2a9d41646461c36176437bdcfe004bded20 WatchSource:0}: Error finding container 2d81ff558c15169b19b213afa552c2a9d41646461c36176437bdcfe004bded20: Status 404 returned error can't find the container with id 2d81ff558c15169b19b213afa552c2a9d41646461c36176437bdcfe004bded20
Jul 16 21:48:55.383779 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.383760    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4hwcf"
Jul 16 21:48:55.389536 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.389518    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-czz97"
Jul 16 21:48:55.390309 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.390289    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d2b7e9_dfe9_4207_b112_d9ca06c42891.slice/crio-f65d41e549cf32847b91b12158aec12eefcd8b7573a9282cef930ffd0c38162a WatchSource:0}: Error finding container f65d41e549cf32847b91b12158aec12eefcd8b7573a9282cef930ffd0c38162a: Status 404 returned error can't find the container with id f65d41e549cf32847b91b12158aec12eefcd8b7573a9282cef930ffd0c38162a
Jul 16 21:48:55.395528 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.395508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6kcq4"
Jul 16 21:48:55.396775 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.396755    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec3f9ce_b783_4b72_a0ea_7ea8bc02e52e.slice/crio-ade695b195ae56c82fe29cf86758d4f91c66528adfd0f412655fcf1521b1d7dc WatchSource:0}: Error finding container ade695b195ae56c82fe29cf86758d4f91c66528adfd0f412655fcf1521b1d7dc: Status 404 returned error can't find the container with id ade695b195ae56c82fe29cf86758d4f91c66528adfd0f412655fcf1521b1d7dc
Jul 16 21:48:55.403640 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.403614    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51e6ee9_1ce5_4258_8ee5_e974a38a07ae.slice/crio-ebbfa58f3d7a99292f69d4f7bd6e7a33ed1de07afb331531590cd7aae4b2e84e WatchSource:0}: Error finding container ebbfa58f3d7a99292f69d4f7bd6e7a33ed1de07afb331531590cd7aae4b2e84e: Status 404 returned error can't find the container with id ebbfa58f3d7a99292f69d4f7bd6e7a33ed1de07afb331531590cd7aae4b2e84e
Jul 16 21:48:55.429013 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.428993    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs"
Jul 16 21:48:55.433748 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.433731    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xc6wt"
Jul 16 21:48:55.437481 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.437461    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9fd0b51_ccba_49b6_8ea2_26dd143d4916.slice/crio-e3fcc6619475af03c4dcb8612e3eef3601a83680a5897a5a3d4e22d7f191a431 WatchSource:0}: Error finding container e3fcc6619475af03c4dcb8612e3eef3601a83680a5897a5a3d4e22d7f191a431: Status 404 returned error can't find the container with id e3fcc6619475af03c4dcb8612e3eef3601a83680a5897a5a3d4e22d7f191a431
Jul 16 21:48:55.441837 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:48:55.441815    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56254e7e_c6df_4831_a4da_c5f1b9e99736.slice/crio-063e34630e5169ca1b48b7c74353cd15cac827177cdd4393b443cba3a08850ac WatchSource:0}: Error finding container 063e34630e5169ca1b48b7c74353cd15cac827177cdd4393b443cba3a08850ac: Status 404 returned error can't find the container with id 063e34630e5169ca1b48b7c74353cd15cac827177cdd4393b443cba3a08850ac
Jul 16 21:48:55.605568 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.605491    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:55.605568 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.605551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:55.605793 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.605743    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:55.605855 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.605829    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:56.605798992 +0000 UTC m=+2.995072757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:55.606424 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.606322    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:55.606424 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.606385    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret podName:5f9adec6-0471-441e-8c76-2dbdd18bfc18 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:56.606367509 +0000 UTC m=+2.995641279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret") pod "global-pull-secret-syncer-zkdf9" (UID: "5f9adec6-0471-441e-8c76-2dbdd18bfc18") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:55.807330 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:55.807291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:55.807533 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.807479    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:48:55.807533 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.807505    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:48:55.807533 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.807521    2564 projected.go:196] Error preparing data for projected volume kube-api-access-tlr2l for pod openshift-network-diagnostics/network-check-target-8ltd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:55.807697 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:55.807584    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l podName:4cbd2da1-d5ea-41f9-a004-dd503966a636 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:56.807563148 +0000 UTC m=+3.196836931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlr2l" (UniqueName: "kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l") pod "network-check-target-8ltd5" (UID: "4cbd2da1-d5ea-41f9-a004-dd503966a636") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:56.128983 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.128862    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerStarted","Data":"ade695b195ae56c82fe29cf86758d4f91c66528adfd0f412655fcf1521b1d7dc"}
Jul 16 21:48:56.144826 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.144748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4hwcf" event={"ID":"22d2b7e9-dfe9-4207-b112-d9ca06c42891","Type":"ContainerStarted","Data":"f65d41e549cf32847b91b12158aec12eefcd8b7573a9282cef930ffd0c38162a"}
Jul 16 21:48:56.148358 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.148322    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:43:55 +0000 UTC" deadline="2028-04-18 02:30:50.593800194 +0000 UTC"
Jul 16 21:48:56.148358 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.148358    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15388h41m54.445446754s"
Jul 16 21:48:56.154425 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.154308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"bae453749cedffbd6b92c7f878ca7154","Type":"ContainerStarted","Data":"65f407c607e051f17df8b5b70fa6021ac1ffdf58c4dc1e5b99e0ba801f439c07"}
Jul 16 21:48:56.158490 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.157903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xc6wt" event={"ID":"56254e7e-c6df-4831-a4da-c5f1b9e99736","Type":"ContainerStarted","Data":"063e34630e5169ca1b48b7c74353cd15cac827177cdd4393b443cba3a08850ac"}
Jul 16 21:48:56.170817 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.170735    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs" event={"ID":"f9fd0b51-ccba-49b6-8ea2-26dd143d4916","Type":"ContainerStarted","Data":"e3fcc6619475af03c4dcb8612e3eef3601a83680a5897a5a3d4e22d7f191a431"}
Jul 16 21:48:56.177809 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.177520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"2d81ff558c15169b19b213afa552c2a9d41646461c36176437bdcfe004bded20"}
Jul 16 21:48:56.183337 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.183305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nwfwh" event={"ID":"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e","Type":"ContainerStarted","Data":"8a3728acb73d713c764d4f283ea647ed346e49705d71aec41f86d1e2a033b1a4"}
Jul 16 21:48:56.185128 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.184990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s8d4d" event={"ID":"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41","Type":"ContainerStarted","Data":"1de362d8611b30161a7d224b65edbdb4d0dba8c5736823dbc4eb9bd833a20e57"}
Jul 16 21:48:56.198075 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.198047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7vc4n" event={"ID":"234d30fa-4935-4b4e-9091-9dbc56829631","Type":"ContainerStarted","Data":"8649009008b3f60da730f8cb139cbb68dfacbe09b070936dadb31d399168795f"}
Jul 16 21:48:56.205515 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.205485    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" event={"ID":"9617a17fc683a92e3fbbd52c642b5794","Type":"ContainerStarted","Data":"bf7c90e72b7ac024810136b59d1aa61c288be0509139e465c117b4aab2ebb404"}
Jul 16 21:48:56.208091 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.208065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6kcq4" event={"ID":"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae","Type":"ContainerStarted","Data":"ebbfa58f3d7a99292f69d4f7bd6e7a33ed1de07afb331531590cd7aae4b2e84e"}
Jul 16 21:48:56.443450 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.443419    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:48:56.615149 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.614322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:56.615149 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.614415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:56.615149 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.614540    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:56.615149 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.614605    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret podName:5f9adec6-0471-441e-8c76-2dbdd18bfc18 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:58.614584504 +0000 UTC m=+5.003858273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret") pod "global-pull-secret-syncer-zkdf9" (UID: "5f9adec6-0471-441e-8c76-2dbdd18bfc18") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:56.615149 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.615058    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:56.615149 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.615114    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:58.615096364 +0000 UTC m=+5.004370148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:56.816164 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:56.816054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:56.816379 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.816267    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:48:56.816379 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.816288    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:48:56.816379 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.816300    2564 projected.go:196] Error preparing data for projected volume kube-api-access-tlr2l for pod openshift-network-diagnostics/network-check-target-8ltd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:56.816379 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:56.816357    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l podName:4cbd2da1-d5ea-41f9-a004-dd503966a636 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:58.816339836 +0000 UTC m=+5.205613617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlr2l" (UniqueName: "kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l") pod "network-check-target-8ltd5" (UID: "4cbd2da1-d5ea-41f9-a004-dd503966a636") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:57.119658 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:57.119572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:57.119834 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:57.119745    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:48:57.121141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:57.120284    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:57.121141 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:57.120386    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:48:57.121141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:57.120455    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:57.121141 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:57.120541    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:48:57.149531 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:57.149463    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:43:55 +0000 UTC" deadline="2028-01-14 06:00:43.797000406 +0000 UTC"
Jul 16 21:48:57.149531 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:57.149522    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13112h11m46.647483015s"
Jul 16 21:48:58.636494 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:58.636453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:58.637057 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:58.636530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:58.637057 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.636659    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:58.637057 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.636724    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:02.636704541 +0000 UTC m=+9.025978310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:48:58.637300 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.637130    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:58.637300 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.637199    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret podName:5f9adec6-0471-441e-8c76-2dbdd18bfc18 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:02.637180911 +0000 UTC m=+9.026454677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret") pod "global-pull-secret-syncer-zkdf9" (UID: "5f9adec6-0471-441e-8c76-2dbdd18bfc18") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:48:58.838672 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:58.838624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:58.838898 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.838873    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:48:58.838898 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.838895    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:48:58.839076 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.838908    2564 projected.go:196] Error preparing data for projected volume kube-api-access-tlr2l for pod openshift-network-diagnostics/network-check-target-8ltd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:58.839076 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:58.838991    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l podName:4cbd2da1-d5ea-41f9-a004-dd503966a636 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:02.838952357 +0000 UTC m=+9.228226126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlr2l" (UniqueName: "kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l") pod "network-check-target-8ltd5" (UID: "4cbd2da1-d5ea-41f9-a004-dd503966a636") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:48:59.119668 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:59.119589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:48:59.119835 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:59.119727    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:48:59.120072 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:59.120043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:48:59.120202 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:59.120170    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:48:59.120267 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:48:59.120219    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:48:59.120320 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:48:59.120295    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:01.119618 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:01.119586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:01.120112 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:01.119730    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:01.120112 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:01.119806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:01.120112 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:01.119829    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:01.120112 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:01.119895    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:01.120112 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:01.120025    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:02.678335 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:02.678285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:02.678796 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:02.678364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:02.678796 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.678469    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:49:02.678796 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.678492    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:49:02.678796 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.678551    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret podName:5f9adec6-0471-441e-8c76-2dbdd18bfc18 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:10.678526515 +0000 UTC m=+17.067800309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret") pod "global-pull-secret-syncer-zkdf9" (UID: "5f9adec6-0471-441e-8c76-2dbdd18bfc18") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:49:02.678796 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.678575    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:10.678563787 +0000 UTC m=+17.067837558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:49:02.880459 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:02.880422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:02.880669 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.880567    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:49:02.880669 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.880591    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:49:02.880669 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.880606    2564 projected.go:196] Error preparing data for projected volume kube-api-access-tlr2l for pod openshift-network-diagnostics/network-check-target-8ltd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:49:02.880669 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:02.880667    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l podName:4cbd2da1-d5ea-41f9-a004-dd503966a636 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:10.880645906 +0000 UTC m=+17.269919675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlr2l" (UniqueName: "kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l") pod "network-check-target-8ltd5" (UID: "4cbd2da1-d5ea-41f9-a004-dd503966a636") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:49:03.119948 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:03.119843    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:03.120131 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:03.120011    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:03.120131 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:03.119843    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:03.120131 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:03.120095    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:03.120131 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:03.120006    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:03.120357 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:03.120192    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:05.119899 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:05.119490    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:05.119899 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:05.119495    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:05.119899 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:05.119596    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:05.119899 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:05.119644    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:05.119899 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:05.119727    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:05.119899 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:05.119889    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:07.119501 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:07.119467    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:07.119501 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:07.119485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:07.119924 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:07.119565    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:07.119924 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:07.119621    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:07.119924 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:07.119711    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:07.119924 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:07.119780    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:09.119318 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:09.119284    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:09.119730 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:09.119393    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:09.119730 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:09.119404    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:09.119730 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:09.119492    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:09.119730 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:09.119528    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:09.119730 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:09.119580    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:10.741830 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:10.741795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:10.742264 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:10.741857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:10.742264 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.741991    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:49:10.742264 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.742007    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:49:10.742264 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.742065    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret podName:5f9adec6-0471-441e-8c76-2dbdd18bfc18 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:26.742046575 +0000 UTC m=+33.131320351 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret") pod "global-pull-secret-syncer-zkdf9" (UID: "5f9adec6-0471-441e-8c76-2dbdd18bfc18") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:49:10.742264 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.742098    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:26.742089778 +0000 UTC m=+33.131363542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:49:10.943414 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:10.943376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:10.943599 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.943533    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:49:10.943599 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.943551    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:49:10.943599 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.943564    2564 projected.go:196] Error preparing data for projected volume kube-api-access-tlr2l for pod openshift-network-diagnostics/network-check-target-8ltd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:49:10.943716 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:10.943628    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l podName:4cbd2da1-d5ea-41f9-a004-dd503966a636 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:26.943609647 +0000 UTC m=+33.332883416 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlr2l" (UniqueName: "kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l") pod "network-check-target-8ltd5" (UID: "4cbd2da1-d5ea-41f9-a004-dd503966a636") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:49:11.119209 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:11.119133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:11.119386 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:11.119133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:11.119386 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:11.119256    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:11.119386 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:11.119141    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:11.119386 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:11.119360    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:11.119598 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:11.119434    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:13.119675 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:13.119635    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:13.119675 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:13.119678    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:13.120283 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:13.119680    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:13.120283 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:13.119764    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:13.120283 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:13.119827    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:13.120283 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:13.119928    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:15.118861 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:15.118810    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:15.119328 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:15.118867    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:15.119328 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:15.118951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:15.119328 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:15.118959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:15.119328 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:15.119031    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:15.119328 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:15.119093    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:17.119090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.118866    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:17.119646 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.118941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:17.119646 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:17.119173    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:17.119646 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.118947    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:17.119646 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:17.119245    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:17.119646 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:17.119344    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:17.259233 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.259207    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s8d4d" event={"ID":"6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41","Type":"ContainerStarted","Data":"ec6ed1478f000ce4aa45b45d92f19940a1fd4185104831aa9732a17d52ce85d2"}
Jul 16 21:49:17.260589 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.260565    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7vc4n" event={"ID":"234d30fa-4935-4b4e-9091-9dbc56829631","Type":"ContainerStarted","Data":"5110936cc132f5613b82e112f85a3fff16b3a560ededb39dd0d4f7ed7f8e78d0"}
Jul 16 21:49:17.261777 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.261759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" event={"ID":"9617a17fc683a92e3fbbd52c642b5794","Type":"ContainerStarted","Data":"e9e0e864354899df27a2d6591c6f5df08a3312989c252afc286b5afa72aac802"}
Jul 16 21:49:17.263030 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.263010    2564 generic.go:356] "Generic (PLEG): container finished" podID="eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e" containerID="b265fec0462c66c23b65504c60210a354b4859bf831d13e1c34973d46b054c0c" exitCode=0
Jul 16 21:49:17.263107 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.263070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerDied","Data":"b265fec0462c66c23b65504c60210a354b4859bf831d13e1c34973d46b054c0c"}
Jul 16 21:49:17.267136 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.266688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4hwcf" event={"ID":"22d2b7e9-dfe9-4207-b112-d9ca06c42891","Type":"ContainerStarted","Data":"221db814f8bce0f5d90817d7c4a4dbe555c4930413ecfcd63beec866c9a052fc"}
Jul 16 21:49:17.270454 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.270430    2564 generic.go:356] "Generic (PLEG): container finished" podID="bae453749cedffbd6b92c7f878ca7154" containerID="9dc34f51e7627199408e5e2a177d8cc48fa041646ffc78b8504e8c7f46161b69" exitCode=0
Jul 16 21:49:17.270534 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.270511    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"bae453749cedffbd6b92c7f878ca7154","Type":"ContainerDied","Data":"9dc34f51e7627199408e5e2a177d8cc48fa041646ffc78b8504e8c7f46161b69"}
Jul 16 21:49:17.272290 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.272269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xc6wt" event={"ID":"56254e7e-c6df-4831-a4da-c5f1b9e99736","Type":"ContainerStarted","Data":"c11c40ab524a8b908d4c456924993751a1d79378296fa95402b8fae9e70e238a"}
Jul 16 21:49:17.273691 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.273671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs" event={"ID":"f9fd0b51-ccba-49b6-8ea2-26dd143d4916","Type":"ContainerStarted","Data":"d27547a0438fa6663141398bf77772f84e3ab69ab8f7fa993a15e78c85d5128a"}
Jul 16 21:49:17.275885 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.275870    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:49:17.276178 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.276162    2564 generic.go:356] "Generic (PLEG): container finished" podID="a215a163-d0cc-42a2-87cb-c50e41d39cfc" containerID="a263d6efad77b7097608dc2d00fe77f31ab4ae09d339f7daa5a16e7ba713070a" exitCode=1
Jul 16 21:49:17.276238 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.276215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"4ca8631d2f79c203fcdfb8e72561862492e089401ace5c86a88513c9da5066b5"}
Jul 16 21:49:17.276288 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.276273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"6353db4f9d15537c5a0d4e136d1fa1a77e1d5455d08cb3c1a15a842a19de45fe"}
Jul 16 21:49:17.276338 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.276326    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"b9af3481d4aa1233595711edef6b9a6edaa77d7eb07e34701e1b56c8b4f22e3a"}
Jul 16 21:49:17.276387 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.276375    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerDied","Data":"a263d6efad77b7097608dc2d00fe77f31ab4ae09d339f7daa5a16e7ba713070a"}
Jul 16 21:49:17.276434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.276422    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"f8482662fc80a0b5693936daadb33b6f6e5ec9466289959a3e6b16980f6a773c"}
Jul 16 21:49:17.277505 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.277487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nwfwh" event={"ID":"e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e","Type":"ContainerStarted","Data":"fdf94e467311e20f950716c9ed7baa9976d9ed352b024481fbf0cb0bcea1a61a"}
Jul 16 21:49:17.333714 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.333496    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4hwcf" podStartSLOduration=1.544959737 podStartE2EDuration="22.333478158s" totalImagesPullingTime="20.788518421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:17.31467973 +0000 UTC m=+23.703953516" watchObservedRunningTime="2026-07-16 21:49:17.333478158 +0000 UTC m=+23.722751944"
Jul 16 21:49:17.365318 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.365276    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7vc4n" podStartSLOduration=11.080674006 podStartE2EDuration="23.365262327s" totalImagesPullingTime="12.284588321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:17.334160963 +0000 UTC m=+23.723434734" watchObservedRunningTime="2026-07-16 21:49:17.365262327 +0000 UTC m=+23.754536113"
Jul 16 21:49:17.403336 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.403294    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xc6wt" podStartSLOduration=1.6650460950000001 podStartE2EDuration="22.4032745s" totalImagesPullingTime="20.738228405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:17.365123692 +0000 UTC m=+23.754397477" watchObservedRunningTime="2026-07-16 21:49:17.4032745 +0000 UTC m=+23.792548287"
Jul 16 21:49:17.465775 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.465724    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" podStartSLOduration=22.465708965 podStartE2EDuration="22.465708965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:17.43655594 +0000 UTC m=+23.825829725" watchObservedRunningTime="2026-07-16 21:49:17.465708965 +0000 UTC m=+23.854982761"
Jul 16 21:49:17.495689 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:17.495638    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s8d4d" podStartSLOduration=2.870874481 podStartE2EDuration="23.495619029s" totalImagesPullingTime="20.624744548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:17.466608881 +0000 UTC m=+23.855882665" watchObservedRunningTime="2026-07-16 21:49:17.495619029 +0000 UTC m=+23.884892818"
Jul 16 21:49:18.282998 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.282963    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6kcq4" event={"ID":"e51e6ee9-1ce5-4258-8ee5-e974a38a07ae","Type":"ContainerStarted","Data":"ed54e45de8eb8742072577a0c22a95a530b88f32848a8941574a613a8f454978"}
Jul 16 21:49:18.285582 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.285551    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"bae453749cedffbd6b92c7f878ca7154","Type":"ContainerStarted","Data":"0216dc7254cd89700f175e477328963e36b338555879e4f866c6f7265eeb628b"}
Jul 16 21:49:18.289717 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.289700    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:49:18.290531 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.290127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"6813609ab8c46527ac79781382c51b9b122b69f94eded4be21ee4134a724e3a3"}
Jul 16 21:49:18.305389 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.305343    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nwfwh" podStartSLOduration=3.655624248 podStartE2EDuration="24.305324161s" totalImagesPullingTime="20.649699913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:17.495864265 +0000 UTC m=+23.885138053" watchObservedRunningTime="2026-07-16 21:49:18.305324161 +0000 UTC m=+24.694597939"
Jul 16 21:49:18.308537 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.307111    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6kcq4" podStartSLOduration=2.533359313 podStartE2EDuration="23.307089035s" totalImagesPullingTime="20.773729722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:18.305877635 +0000 UTC m=+24.695151419" watchObservedRunningTime="2026-07-16 21:49:18.307089035 +0000 UTC m=+24.696362822"
Jul 16 21:49:18.327479 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.327428    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" podStartSLOduration=23.327410465 podStartE2EDuration="23.327410465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:18.327126824 +0000 UTC m=+24.716400616" watchObservedRunningTime="2026-07-16 21:49:18.327410465 +0000 UTC m=+24.716684251"
Jul 16 21:49:18.334239 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:18.334218    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 21:49:19.060464 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.059700    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T21:49:18.334234384Z","UUID":"1b3cb3ca-815f-41d3-af74-1320c31fab38","Handler":null,"Name":"","Endpoint":""}
Jul 16 21:49:19.061496 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.061474    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 21:49:19.061607 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.061520    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 21:49:19.118892 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.118856    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:19.119130 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.119007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:19.119130 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:19.119026    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:19.119130 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:19.119099    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:19.119303 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.119130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:19.119303 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:19.119232    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:19.294452 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.294410    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs" event={"ID":"f9fd0b51-ccba-49b6-8ea2-26dd143d4916","Type":"ContainerStarted","Data":"d0eed8aa501989859c861d49373ae5cc9d9bcf3e40dcdd659149d8a72300fb80"}
Jul 16 21:49:19.631542 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:19.631521    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:49:20.298669 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:20.298631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs" event={"ID":"f9fd0b51-ccba-49b6-8ea2-26dd143d4916","Type":"ContainerStarted","Data":"83dd2cb1140dba2487c61d4f90d42db03a850b2888409f6fba1f0db32f2dc35d"}
Jul 16 21:49:20.302118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:20.302094    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:49:20.302506 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:20.302483    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"c3186b258ac0f63c33c922ec67a148c14cda8be49bc3a18232192803b476dfd5"}
Jul 16 21:49:21.119231 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:21.119194    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:21.119411 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:21.119329    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:21.119411 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:21.119345    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:21.119411 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:21.119363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:21.119582 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:21.119458    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:21.119582 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:21.119541    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:21.233598 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:21.233563    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:49:21.234334 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:21.234317    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:49:21.252582 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:21.252525    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wf2vs" podStartSLOduration=3.448560711 podStartE2EDuration="26.252505896s" totalImagesPullingTime="22.803945185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:20.322248097 +0000 UTC m=+26.711521896" watchObservedRunningTime="2026-07-16 21:49:21.252505896 +0000 UTC m=+27.641779682"
Jul 16 21:49:21.305671 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:21.305644    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7vc4n"
Jul 16 21:49:22.310148 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:22.309971    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:49:22.311031 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:22.310496    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"0eb10e77b04fc26c3d0c6602c41625d1c96e91d3ab08695c8523dcab2f5a4d65"}
Jul 16 21:49:22.311031 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:22.310764    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:49:22.311031 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:22.310900    2564 scope.go:117] "RemoveContainer" containerID="a263d6efad77b7097608dc2d00fe77f31ab4ae09d339f7daa5a16e7ba713070a"
Jul 16 21:49:22.312545 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:22.312518    2564 generic.go:356] "Generic (PLEG): container finished" podID="eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e" containerID="17249523177bb37ab8f41cc5163b0a91c7f8457d6c44d7d615159dd93244062d" exitCode=0
Jul 16 21:49:22.312651 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:22.312600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerDied","Data":"17249523177bb37ab8f41cc5163b0a91c7f8457d6c44d7d615159dd93244062d"}
Jul 16 21:49:22.327734 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:22.327714    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:49:23.120011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.119804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:23.120271 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.119831    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:23.120271 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:23.120107    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:23.120271 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.119876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:23.120271 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:23.120195    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:23.120485 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:23.120274    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:23.317704 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.317679    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:49:23.318178 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.318073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" event={"ID":"a215a163-d0cc-42a2-87cb-c50e41d39cfc","Type":"ContainerStarted","Data":"a6891d485cb0e7feb421715dd5d54813d17755096a4dbe96012329a4aaab2b0d"}
Jul 16 21:49:23.318245 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.318185    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 16 21:49:23.318474 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.318457    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:49:23.333551 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.333529    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:49:23.351753 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:23.351714    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd" podStartSLOduration=7.075602975 podStartE2EDuration="28.351702171s" totalImagesPullingTime="21.276099196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:23.350701804 +0000 UTC m=+29.739975632" watchObservedRunningTime="2026-07-16 21:49:23.351702171 +0000 UTC m=+29.740975956"
Jul 16 21:49:24.055478 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.055433    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:49:24.069999 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.069974    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zkdf9"]
Jul 16 21:49:24.070129 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.070054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:24.070166 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:24.070127    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:24.073170 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.073143    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ltd5"]
Jul 16 21:49:24.073302 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.073280    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:24.073402 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:24.073379    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:24.073852 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.073833    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hzwzf"]
Jul 16 21:49:24.073953 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.073941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:24.074063 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:24.074042    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:24.322396 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.322320    2564 generic.go:356] "Generic (PLEG): container finished" podID="eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e" containerID="6cafb1d9ef6ce062d28ccc738250d00961e0f50364ccb97a08be17c5373ec56d" exitCode=0
Jul 16 21:49:24.322782 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:24.322408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerDied","Data":"6cafb1d9ef6ce062d28ccc738250d00961e0f50364ccb97a08be17c5373ec56d"}
Jul 16 21:49:26.119896 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.119821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:26.119896 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.119876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:26.119896 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.119888    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:26.120331 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.119996    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:26.120331 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.120035    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:26.120331 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.120106    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:26.328780 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.328750    2564 generic.go:356] "Generic (PLEG): container finished" podID="eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e" containerID="a3e25ff3ee08c555a4a966aa4d510371659a938b566b5881881188451b70139a" exitCode=0
Jul 16 21:49:26.328976 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.328794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerDied","Data":"a3e25ff3ee08c555a4a966aa4d510371659a938b566b5881881188451b70139a"}
Jul 16 21:49:26.760728 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.760699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:26.760879 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.760740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:26.760879 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.760851    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:49:26.760879 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.760851    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:49:26.761001 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.760924    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret podName:5f9adec6-0471-441e-8c76-2dbdd18bfc18 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:58.76089297 +0000 UTC m=+65.150166734 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret") pod "global-pull-secret-syncer-zkdf9" (UID: "5f9adec6-0471-441e-8c76-2dbdd18bfc18") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:49:26.761001 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.760938    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:58.760931601 +0000 UTC m=+65.150205366 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:49:26.963028 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:26.963002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:26.963166 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.963115    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:49:26.963166 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.963129    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:49:26.963166 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.963137    2564 projected.go:196] Error preparing data for projected volume kube-api-access-tlr2l for pod openshift-network-diagnostics/network-check-target-8ltd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:49:26.963266 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:26.963180    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l podName:4cbd2da1-d5ea-41f9-a004-dd503966a636 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:58.96316696 +0000 UTC m=+65.352440724 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlr2l" (UniqueName: "kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l") pod "network-check-target-8ltd5" (UID: "4cbd2da1-d5ea-41f9-a004-dd503966a636") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:49:28.119660 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:28.119626    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:28.120042 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:28.119627    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:28.120042 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:28.119731    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:28.120042 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:28.119748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:28.120042 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:28.119822    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:28.120042 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:28.119876    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:30.119726 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.119696    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:30.119726 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.119731    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:30.120240 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.119695    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:30.120240 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.119817    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ltd5" podUID="4cbd2da1-d5ea-41f9-a004-dd503966a636"
Jul 16 21:49:30.120240 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.119959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hzwzf" podUID="9d31ef25-6751-41d8-9583-11a3455909e6"
Jul 16 21:49:30.120240 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.120047    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zkdf9" podUID="5f9adec6-0471-441e-8c76-2dbdd18bfc18"
Jul 16 21:49:30.412790 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.412722    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeReady"
Jul 16 21:49:30.412994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.412886    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 21:49:30.473303 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.472798    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-758897b654-n8j2k"]
Jul 16 21:49:30.476734 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.476711    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"]
Jul 16 21:49:30.477040 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.477004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.479577 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.479553    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"]
Jul 16 21:49:30.481752 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.480958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.481752 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.481029    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mdgnt\""
Jul 16 21:49:30.481752 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.481060    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 21:49:30.481752 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.481119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 21:49:30.483141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.483120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"
Jul 16 21:49:30.483397 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.483382    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 21:49:30.484446 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.483525    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-77j96"]
Jul 16 21:49:30.484568 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.484451    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.484650 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.484570    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.485014 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.484851    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sf8vj\""
Jul 16 21:49:30.485373 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.485354    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 16 21:49:30.486204 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.486167    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-trdvm"]
Jul 16 21:49:30.486435 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.486398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.488093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.487627    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.488093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.487849    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kbkv6\""
Jul 16 21:49:30.488093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.487863    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.488962 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.488941    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"]
Jul 16 21:49:30.489047 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.489015    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.489274 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.489240    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 16 21:49:30.489712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.489496    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-sdp5m\""
Jul 16 21:49:30.489822 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.489785    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 16 21:49:30.489950 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.489859    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.490339 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.490317    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 16 21:49:30.490989 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.490970    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 21:49:30.491521 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.491504    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"]
Jul 16 21:49:30.491624 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.491584    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:30.493948 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.493877    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64db4fc64d-wlpxp"]
Jul 16 21:49:30.494028 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.493966    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.494324 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.494306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.497050 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.497028    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"]
Jul 16 21:49:30.497942 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.497904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.499434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499414    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 16 21:49:30.499535 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499492    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 16 21:49:30.499535 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499514    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.499670 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499655    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.499751 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.500060 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499840    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.500060 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499973    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-k9m52\""
Jul 16 21:49:30.500060 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499982    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 16 21:49:30.500060 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.499986    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 16 21:49:30.500060 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.500023    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.500310 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.500294    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-csql9\""
Jul 16 21:49:30.500639 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.500430    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 16 21:49:30.500639 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.500446    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.500639 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.500520    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.500799 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.500776    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ftbgv\""
Jul 16 21:49:30.506864 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.506843    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 16 21:49:30.507094 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.507077    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.507159 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.507123    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 16 21:49:30.507215 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.507159    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 16 21:49:30.507268 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.507254    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.507337 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.507127    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.507412 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.507397    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 16 21:49:30.507412 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.507404    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wdv82\""
Jul 16 21:49:30.508248 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.508219    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 16 21:49:30.508352 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.508308    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5gkfj\""
Jul 16 21:49:30.508461 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.508229    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 16 21:49:30.508555 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.508539    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.512546 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.512518    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 16 21:49:30.519518 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.519495    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-77j96"]
Jul 16 21:49:30.520045 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.520025    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 16 21:49:30.520144 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.520132    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"]
Jul 16 21:49:30.521525 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.521506    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"]
Jul 16 21:49:30.521748 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.521727    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wn4d2"]
Jul 16 21:49:30.524660 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.524640    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"]
Jul 16 21:49:30.525779 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.525757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.528364 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.528333    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8pnfs"]
Jul 16 21:49:30.528483 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.528465    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:30.530552 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.530522    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 21:49:30.530782 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.530729    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 21:49:30.530782 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.530753    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5kfkb\""
Jul 16 21:49:30.531495 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.531474    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"]
Jul 16 21:49:30.531582 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.531514    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"]
Jul 16 21:49:30.531725 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.531705    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pwsnw\""
Jul 16 21:49:30.531845 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.531830    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.532030 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.532013    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 16 21:49:30.532405 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.532368    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 16 21:49:30.534629 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.534612    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64db4fc64d-wlpxp"]
Jul 16 21:49:30.534759 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.534648    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"]
Jul 16 21:49:30.534759 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.534679    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758897b654-n8j2k"]
Jul 16 21:49:30.534759 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.534759    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"
Jul 16 21:49:30.535331 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.535312    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.535567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.535553    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.535816 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.535799    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vlpp9\""
Jul 16 21:49:30.535883 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.535815    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 21:49:30.536849 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.536817    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 21:49:30.537076 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.537057    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 21:49:30.537270 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.537254    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2mwsn\""
Jul 16 21:49:30.543940 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.543810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:30.548733 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.548598    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"]
Jul 16 21:49:30.549373 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.549348    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wn4d2"]
Jul 16 21:49:30.558968 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.558948    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8pnfs"]
Jul 16 21:49:30.559712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.559696    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"]
Jul 16 21:49:30.560707 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.560688    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-trdvm"]
Jul 16 21:49:30.573057 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.573037    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"]
Jul 16 21:49:30.590659 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590635    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-589bs\" (UniqueName: \"kubernetes.io/projected/98296b74-f3c7-4a28-aba5-fd0bfab5d32f-kube-api-access-589bs\") pod \"volume-data-source-validator-78cf4c6f6-r4frb\" (UID: \"98296b74-f3c7-4a28-aba5-fd0bfab5d32f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"
Jul 16 21:49:30.590748 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52af9bbb-a233-445a-98e6-22f0cc478c60-config\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.590748 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-trusted-ca\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.590852 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-installation-pull-secrets\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.590852 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1173fb03-41c1-4aad-902f-7b7f6d9bd371-service-ca-bundle\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.590852 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590825    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-default-certificate\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.591011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.591011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wj5c\" (UniqueName: \"kubernetes.io/projected/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-kube-api-access-5wj5c\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:30.591011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.590967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-stats-auth\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.591149 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.591149 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l4tv\" (UniqueName: \"kubernetes.io/projected/52af9bbb-a233-445a-98e6-22f0cc478c60-kube-api-access-5l4tv\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.591149 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmzk\" (UniqueName: \"kubernetes.io/projected/d70792ad-fa1c-4f45-be6c-05892fd3fc19-kube-api-access-sfmzk\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.591149 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591125    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84797633-98b5-4ce4-90ac-779d24813acf-ca-trust-extracted\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.591306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-registry-certificates\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.591306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591195    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1173fb03-41c1-4aad-902f-7b7f6d9bd371-serving-cert\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.591306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:30.591306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1173fb03-41c1-4aad-902f-7b7f6d9bd371-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591314    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvpgj\" (UniqueName: \"kubernetes.io/projected/f6340ec6-1b89-4f62-a863-22358fc973ea-kube-api-access-dvpgj\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb9b083e-7066-40e4-bce1-366de4032eaa-trusted-ca\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591356    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591371    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1173fb03-41c1-4aad-902f-7b7f6d9bd371-tmp\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-image-registry-private-configuration\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591420    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-bound-sa-token\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.591466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb9b083e-7066-40e4-bce1-366de4032eaa-config\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k9hj\" (UniqueName: \"kubernetes.io/projected/cb9b083e-7066-40e4-bce1-366de4032eaa-kube-api-access-7k9hj\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jdjp\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-kube-api-access-6jdjp\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb9b083e-7066-40e4-bce1-366de4032eaa-serving-cert\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/748c76b0-d147-4d51-9377-9409e9faefa9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1173fb03-41c1-4aad-902f-7b7f6d9bd371-snapshots\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591659    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f6340ec6-1b89-4f62-a863-22358fc973ea-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52af9bbb-a233-445a-98e6-22f0cc478c60-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxv7g\" (UniqueName: \"kubernetes.io/projected/1173fb03-41c1-4aad-902f-7b7f6d9bd371-kube-api-access-dxv7g\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/748c76b0-d147-4d51-9377-9409e9faefa9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.591820 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.591761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwf79\" (UniqueName: \"kubernetes.io/projected/748c76b0-d147-4d51-9377-9409e9faefa9-kube-api-access-hwf79\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.693074 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693040    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-stats-auth\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.693247 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.693247 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.693247 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.693205    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:30.693247 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l4tv\" (UniqueName: \"kubernetes.io/projected/52af9bbb-a233-445a-98e6-22f0cc478c60-kube-api-access-5l4tv\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.693451 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.693281    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls podName:f6340ec6-1b89-4f62-a863-22358fc973ea nodeName:}" failed. No retries permitted until 2026-07-16 21:49:31.193260796 +0000 UTC m=+37.582534564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2f7rl" (UID: "f6340ec6-1b89-4f62-a863-22358fc973ea") : secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:30.693451 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e635ee7-8ae6-4a90-a66f-e33cfd434838-config-volume\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.693451 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmzk\" (UniqueName: \"kubernetes.io/projected/d70792ad-fa1c-4f45-be6c-05892fd3fc19-kube-api-access-sfmzk\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.693451 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84797633-98b5-4ce4-90ac-779d24813acf-ca-trust-extracted\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.693652 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-registry-certificates\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.693652 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693563    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1173fb03-41c1-4aad-902f-7b7f6d9bd371-serving-cert\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.693652 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693590    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:30.693652 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693626    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1173fb03-41c1-4aad-902f-7b7f6d9bd371-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.693652 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693647    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvpgj\" (UniqueName: \"kubernetes.io/projected/f6340ec6-1b89-4f62-a863-22358fc973ea-kube-api-access-dvpgj\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.693879 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88805df0-8f06-432d-9316-c97536c8bc08-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:30.693879 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693735    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84797633-98b5-4ce4-90ac-779d24813acf-ca-trust-extracted\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.693879 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.693759    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:49:30.693879 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.693812    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls podName:b61aa2f1-c376-4a76-a60b-0ba45afdbd66 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:31.193794839 +0000 UTC m=+37.583068626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8dg4p" (UID: "b61aa2f1-c376-4a76-a60b-0ba45afdbd66") : secret "samples-operator-tls" not found
Jul 16 21:49:30.693879 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vswvp\" (UniqueName: \"kubernetes.io/projected/512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae-kube-api-access-vswvp\") pod \"network-check-source-7bc5b6bf85-pwrkk\" (UID: \"512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"
Jul 16 21:49:30.693879 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb9b083e-7066-40e4-bce1-366de4032eaa-trusted-ca\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1173fb03-41c1-4aad-902f-7b7f6d9bd371-tmp\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.693973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-image-registry-private-configuration\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.694005    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hswrp\" (UniqueName: \"kubernetes.io/projected/2e635ee7-8ae6-4a90-a66f-e33cfd434838-kube-api-access-hswrp\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.694033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.694055    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.694060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-bound-sa-token\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.694069    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758897b654-n8j2k: secret "image-registry-tls" not found
Jul 16 21:49:30.694197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.694093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb9b083e-7066-40e4-bce1-366de4032eaa-config\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.695222    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls podName:84797633-98b5-4ce4-90ac-779d24813acf nodeName:}" failed. No retries permitted until 2026-07-16 21:49:31.194104578 +0000 UTC m=+37.583378346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls") pod "image-registry-758897b654-n8j2k" (UID: "84797633-98b5-4ce4-90ac-779d24813acf") : secret "image-registry-tls" not found
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.695260    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6fnq\" (UniqueName: \"kubernetes.io/projected/3b323658-e5c3-4845-9038-a8695bfed36f-kube-api-access-m6fnq\") pod \"ingress-canary-8pnfs\" (UID: \"3b323658-e5c3-4845-9038-a8695bfed36f\") " pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.695326    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:31.19530504 +0000 UTC m=+37.584578804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : configmap references non-existent config key: service-ca.crt
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.695327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k9hj\" (UniqueName: \"kubernetes.io/projected/cb9b083e-7066-40e4-bce1-366de4032eaa-kube-api-access-7k9hj\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.695382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jdjp\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-kube-api-access-6jdjp\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.695711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb9b083e-7066-40e4-bce1-366de4032eaa-config\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.694126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-registry-certificates\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.695795    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1173fb03-41c1-4aad-902f-7b7f6d9bd371-tmp\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696137    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb9b083e-7066-40e4-bce1-366de4032eaa-trusted-ca\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1173fb03-41c1-4aad-902f-7b7f6d9bd371-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.696284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb9b083e-7066-40e4-bce1-366de4032eaa-serving-cert\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/748c76b0-d147-4d51-9377-9409e9faefa9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1173fb03-41c1-4aad-902f-7b7f6d9bd371-snapshots\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f6340ec6-1b89-4f62-a863-22358fc973ea-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696445    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52af9bbb-a233-445a-98e6-22f0cc478c60-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696502    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxv7g\" (UniqueName: \"kubernetes.io/projected/1173fb03-41c1-4aad-902f-7b7f6d9bd371-kube-api-access-dxv7g\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696532    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/748c76b0-d147-4d51-9377-9409e9faefa9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwf79\" (UniqueName: \"kubernetes.io/projected/748c76b0-d147-4d51-9377-9409e9faefa9-kube-api-access-hwf79\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-589bs\" (UniqueName: \"kubernetes.io/projected/98296b74-f3c7-4a28-aba5-fd0bfab5d32f-kube-api-access-589bs\") pod \"volume-data-source-validator-78cf4c6f6-r4frb\" (UID: \"98296b74-f3c7-4a28-aba5-fd0bfab5d32f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696833    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52af9bbb-a233-445a-98e6-22f0cc478c60-config\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.696875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-trusted-ca\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696934    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1173fb03-41c1-4aad-902f-7b7f6d9bd371-snapshots\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.696939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-installation-pull-secrets\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.697019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e635ee7-8ae6-4a90-a66f-e33cfd434838-tmp-dir\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.697047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b323658-e5c3-4845-9038-a8695bfed36f-cert\") pod \"ingress-canary-8pnfs\" (UID: \"3b323658-e5c3-4845-9038-a8695bfed36f\") " pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.697112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1173fb03-41c1-4aad-902f-7b7f6d9bd371-service-ca-bundle\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.697153    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-default-certificate\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.697181    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.697220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wj5c\" (UniqueName: \"kubernetes.io/projected/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-kube-api-access-5wj5c\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:30.697404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.697348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f6340ec6-1b89-4f62-a863-22358fc973ea-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.698555 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.698059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/748c76b0-d147-4d51-9377-9409e9faefa9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.698555 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.698156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1173fb03-41c1-4aad-902f-7b7f6d9bd371-service-ca-bundle\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.698876 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.698836    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:49:30.698977 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.698901    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:31.1988769 +0000 UTC m=+37.588150663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : secret "router-metrics-certs-default" not found
Jul 16 21:49:30.699388 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.699362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1173fb03-41c1-4aad-902f-7b7f6d9bd371-serving-cert\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.699467 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.699401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52af9bbb-a233-445a-98e6-22f0cc478c60-config\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.700260 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.700235    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-trusted-ca\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.700521 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.700499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-stats-auth\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.700796 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.700543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb9b083e-7066-40e4-bce1-366de4032eaa-serving-cert\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.701085 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.701062    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-image-registry-private-configuration\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.701773 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.701749    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52af9bbb-a233-445a-98e6-22f0cc478c60-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.703170 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.702213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-default-certificate\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.703170 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.702846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/748c76b0-d147-4d51-9377-9409e9faefa9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.706504 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.706479    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-installation-pull-secrets\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.708879 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.708841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l4tv\" (UniqueName: \"kubernetes.io/projected/52af9bbb-a233-445a-98e6-22f0cc478c60-kube-api-access-5l4tv\") pod \"service-ca-operator-5bcbc94c78-jr92b\" (UID: \"52af9bbb-a233-445a-98e6-22f0cc478c60\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.714970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.714924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-bound-sa-token\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.715761 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.715604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmzk\" (UniqueName: \"kubernetes.io/projected/d70792ad-fa1c-4f45-be6c-05892fd3fc19-kube-api-access-sfmzk\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:30.716249 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.716225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k9hj\" (UniqueName: \"kubernetes.io/projected/cb9b083e-7066-40e4-bce1-366de4032eaa-kube-api-access-7k9hj\") pod \"console-operator-565b7b569d-77j96\" (UID: \"cb9b083e-7066-40e4-bce1-366de4032eaa\") " pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.716719 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.716693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wj5c\" (UniqueName: \"kubernetes.io/projected/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-kube-api-access-5wj5c\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:30.717174 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.717122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwf79\" (UniqueName: \"kubernetes.io/projected/748c76b0-d147-4d51-9377-9409e9faefa9-kube-api-access-hwf79\") pod \"kube-storage-version-migrator-operator-7658d9b749-qxdl4\" (UID: \"748c76b0-d147-4d51-9377-9409e9faefa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.718512 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.718483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvpgj\" (UniqueName: \"kubernetes.io/projected/f6340ec6-1b89-4f62-a863-22358fc973ea-kube-api-access-dvpgj\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:30.718709 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.718691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-589bs\" (UniqueName: \"kubernetes.io/projected/98296b74-f3c7-4a28-aba5-fd0bfab5d32f-kube-api-access-589bs\") pod \"volume-data-source-validator-78cf4c6f6-r4frb\" (UID: \"98296b74-f3c7-4a28-aba5-fd0bfab5d32f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"
Jul 16 21:49:30.725850 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.725733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jdjp\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-kube-api-access-6jdjp\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:30.728688 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.728666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxv7g\" (UniqueName: \"kubernetes.io/projected/1173fb03-41c1-4aad-902f-7b7f6d9bd371-kube-api-access-dxv7g\") pod \"insights-operator-555d9d7885-trdvm\" (UID: \"1173fb03-41c1-4aad-902f-7b7f6d9bd371\") " pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.772854 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.772808    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:30.797866 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.797832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.798028 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.797876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e635ee7-8ae6-4a90-a66f-e33cfd434838-config-volume\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.798028 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.797962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88805df0-8f06-432d-9316-c97536c8bc08-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:30.798028 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.797988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vswvp\" (UniqueName: \"kubernetes.io/projected/512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae-kube-api-access-vswvp\") pod \"network-check-source-7bc5b6bf85-pwrkk\" (UID: \"512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"
Jul 16 21:49:30.798028 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.798008    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:49:30.798231 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.798032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hswrp\" (UniqueName: \"kubernetes.io/projected/2e635ee7-8ae6-4a90-a66f-e33cfd434838-kube-api-access-hswrp\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.798231 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.798076    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls podName:2e635ee7-8ae6-4a90-a66f-e33cfd434838 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:31.298055121 +0000 UTC m=+37.687328887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls") pod "dns-default-wn4d2" (UID: "2e635ee7-8ae6-4a90-a66f-e33cfd434838") : secret "dns-default-metrics-tls" not found
Jul 16 21:49:30.798231 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.798118    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6fnq\" (UniqueName: \"kubernetes.io/projected/3b323658-e5c3-4845-9038-a8695bfed36f-kube-api-access-m6fnq\") pod \"ingress-canary-8pnfs\" (UID: \"3b323658-e5c3-4845-9038-a8695bfed36f\") " pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.798231 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.798179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:30.798488 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.798278    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 21:49:30.798488 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.798334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e635ee7-8ae6-4a90-a66f-e33cfd434838-tmp-dir\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.798488 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:30.798371    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert podName:88805df0-8f06-432d-9316-c97536c8bc08 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:31.298349064 +0000 UTC m=+37.687622832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-nj2nn" (UID: "88805df0-8f06-432d-9316-c97536c8bc08") : secret "networking-console-plugin-cert" not found
Jul 16 21:49:30.798488 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.798413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b323658-e5c3-4845-9038-a8695bfed36f-cert\") pod \"ingress-canary-8pnfs\" (UID: \"3b323658-e5c3-4845-9038-a8695bfed36f\") " pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.798678 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.798564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e635ee7-8ae6-4a90-a66f-e33cfd434838-config-volume\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.798678 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.798609    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e635ee7-8ae6-4a90-a66f-e33cfd434838-tmp-dir\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.799206 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.799184    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88805df0-8f06-432d-9316-c97536c8bc08-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:30.801055 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.801034    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b323658-e5c3-4845-9038-a8695bfed36f-cert\") pod \"ingress-canary-8pnfs\" (UID: \"3b323658-e5c3-4845-9038-a8695bfed36f\") " pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.809823 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.809791    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"
Jul 16 21:49:30.820060 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.820039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vswvp\" (UniqueName: \"kubernetes.io/projected/512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae-kube-api-access-vswvp\") pod \"network-check-source-7bc5b6bf85-pwrkk\" (UID: \"512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"
Jul 16 21:49:30.820164 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.820150    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:30.821265 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.821245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6fnq\" (UniqueName: \"kubernetes.io/projected/3b323658-e5c3-4845-9038-a8695bfed36f-kube-api-access-m6fnq\") pod \"ingress-canary-8pnfs\" (UID: \"3b323658-e5c3-4845-9038-a8695bfed36f\") " pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.822141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.822118    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hswrp\" (UniqueName: \"kubernetes.io/projected/2e635ee7-8ae6-4a90-a66f-e33cfd434838-kube-api-access-hswrp\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:30.828415 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.828396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-trdvm"
Jul 16 21:49:30.854565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.854543    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"
Jul 16 21:49:30.866942 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.866924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"
Jul 16 21:49:30.880136 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.880109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8pnfs"
Jul 16 21:49:30.886216 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:30.886192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"
Jul 16 21:49:31.072368 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.072285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:31.147346 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.146966    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.203349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.203596    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.203673    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls podName:b61aa2f1-c376-4a76-a60b-0ba45afdbd66 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:32.203651406 +0000 UTC m=+38.592925187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8dg4p" (UID: "b61aa2f1-c376-4a76-a60b-0ba45afdbd66") : secret "samples-operator-tls" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.204252    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.204320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.204473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.204512    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204702    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204731    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204722    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204773    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758897b654-n8j2k: secret "image-registry-tls" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204758    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls podName:f6340ec6-1b89-4f62-a863-22358fc973ea nodeName:}" failed. No retries permitted until 2026-07-16 21:49:32.204740238 +0000 UTC m=+38.594014021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2f7rl" (UID: "f6340ec6-1b89-4f62-a863-22358fc973ea") : secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204833    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:32.204809318 +0000 UTC m=+38.594083085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : secret "router-metrics-certs-default" not found
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204849    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:32.204839126 +0000 UTC m=+38.594112895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : configmap references non-existent config key: service-ca.crt
Jul 16 21:49:31.205090 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.204862    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls podName:84797633-98b5-4ce4-90ac-779d24813acf nodeName:}" failed. No retries permitted until 2026-07-16 21:49:32.204854029 +0000 UTC m=+38.594127799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls") pod "image-registry-758897b654-n8j2k" (UID: "84797633-98b5-4ce4-90ac-779d24813acf") : secret "image-registry-tls" not found
Jul 16 21:49:31.305493 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.305462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:31.305647 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.305625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:31.305945 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.305828    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 21:49:31.305945 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.305891    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert podName:88805df0-8f06-432d-9316-c97536c8bc08 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:32.305871593 +0000 UTC m=+38.695145380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-nj2nn" (UID: "88805df0-8f06-432d-9316-c97536c8bc08") : secret "networking-console-plugin-cert" not found
Jul 16 21:49:31.306338 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.306319    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:49:31.306398 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:31.306374    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls podName:2e635ee7-8ae6-4a90-a66f-e33cfd434838 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:32.306359642 +0000 UTC m=+38.695633408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls") pod "dns-default-wn4d2" (UID: "2e635ee7-8ae6-4a90-a66f-e33cfd434838") : secret "dns-default-metrics-tls" not found
Jul 16 21:49:31.329670 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.329643    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4"]
Jul 16 21:49:31.332230 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:31.332206    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748c76b0_d147_4d51_9377_9409e9faefa9.slice/crio-be10aa7d842195dacc9e0320fe210fddfc4bf187ca766a2051daeb831eb31352 WatchSource:0}: Error finding container be10aa7d842195dacc9e0320fe210fddfc4bf187ca766a2051daeb831eb31352: Status 404 returned error can't find the container with id be10aa7d842195dacc9e0320fe210fddfc4bf187ca766a2051daeb831eb31352
Jul 16 21:49:31.339653 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.339459    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b"]
Jul 16 21:49:31.343651 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:31.343625    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52af9bbb_a233_445a_98e6_22f0cc478c60.slice/crio-2daa51a1910c09cb7fb01262362a0c60c3e1e649d1e76fff09f8320e8361b0c7 WatchSource:0}: Error finding container 2daa51a1910c09cb7fb01262362a0c60c3e1e649d1e76fff09f8320e8361b0c7: Status 404 returned error can't find the container with id 2daa51a1910c09cb7fb01262362a0c60c3e1e649d1e76fff09f8320e8361b0c7
Jul 16 21:49:31.344243 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.344202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4" event={"ID":"748c76b0-d147-4d51-9377-9409e9faefa9","Type":"ContainerStarted","Data":"be10aa7d842195dacc9e0320fe210fddfc4bf187ca766a2051daeb831eb31352"}
Jul 16 21:49:31.350938 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.350843    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk"]
Jul 16 21:49:31.360002 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:31.359972    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512bb7d4_6c82_4e95_aa19_4f0c2f7f4bae.slice/crio-37d688a2302e88875c6e9fc75e94f4c30fc41fe2b3d41b3bb3d875a358900933 WatchSource:0}: Error finding container 37d688a2302e88875c6e9fc75e94f4c30fc41fe2b3d41b3bb3d875a358900933: Status 404 returned error can't find the container with id 37d688a2302e88875c6e9fc75e94f4c30fc41fe2b3d41b3bb3d875a358900933
Jul 16 21:49:31.360692 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.360674    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-77j96"]
Jul 16 21:49:31.365439 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:31.365414    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9b083e_7066_40e4_bce1_366de4032eaa.slice/crio-ed2f6a00f973552193745827be9cfb8970e1cab3fb8948431b42741ddfb01dcb WatchSource:0}: Error finding container ed2f6a00f973552193745827be9cfb8970e1cab3fb8948431b42741ddfb01dcb: Status 404 returned error can't find the container with id ed2f6a00f973552193745827be9cfb8970e1cab3fb8948431b42741ddfb01dcb
Jul 16 21:49:31.372755 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.372733    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8pnfs"]
Jul 16 21:49:31.376023 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:31.375990    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b323658_e5c3_4845_9038_a8695bfed36f.slice/crio-2345946374fe4c30efacce62cd203b3a8d016dddd11f35529eaea4cdea589986 WatchSource:0}: Error finding container 2345946374fe4c30efacce62cd203b3a8d016dddd11f35529eaea4cdea589986: Status 404 returned error can't find the container with id 2345946374fe4c30efacce62cd203b3a8d016dddd11f35529eaea4cdea589986
Jul 16 21:49:31.381781 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.381758    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb"]
Jul 16 21:49:31.387522 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.387505    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-trdvm"]
Jul 16 21:49:31.390165 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:31.390104    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1173fb03_41c1_4aad_902f_7b7f6d9bd371.slice/crio-4d2bf087f618c07602ff0688a8f4afb3dc7a2a44c7b052bc9559a80390f34343 WatchSource:0}: Error finding container 4d2bf087f618c07602ff0688a8f4afb3dc7a2a44c7b052bc9559a80390f34343: Status 404 returned error can't find the container with id 4d2bf087f618c07602ff0688a8f4afb3dc7a2a44c7b052bc9559a80390f34343
Jul 16 21:49:31.727244 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:31.727214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:32.102434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.102352    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:32.124029 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.122698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:32.124029 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.123631    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:32.127214 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.124455    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:32.129158 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.128938    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xjnvp\""
Jul 16 21:49:32.131838 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.131203    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xnvp2\""
Jul 16 21:49:32.131838 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.131458    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:49:32.131838 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.131589    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:49:32.214314 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.214280    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:32.215038 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.214339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:32.215038 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.214392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:32.215038 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.214420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:32.215038 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.214452    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:32.215038 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.214631    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:34.214588205 +0000 UTC m=+40.603861969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : configmap references non-existent config key: service-ca.crt
Jul 16 21:49:32.215308 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215124    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:49:32.215308 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215177    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:34.215160094 +0000 UTC m=+40.604433865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : secret "router-metrics-certs-default" not found
Jul 16 21:49:32.215308 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215228    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:32.215308 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215260    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls podName:f6340ec6-1b89-4f62-a863-22358fc973ea nodeName:}" failed. No retries permitted until 2026-07-16 21:49:34.215249346 +0000 UTC m=+40.604523116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2f7rl" (UID: "f6340ec6-1b89-4f62-a863-22358fc973ea") : secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:32.215308 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215310    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:49:32.215540 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215338    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls podName:b61aa2f1-c376-4a76-a60b-0ba45afdbd66 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:34.215329339 +0000 UTC m=+40.604603106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8dg4p" (UID: "b61aa2f1-c376-4a76-a60b-0ba45afdbd66") : secret "samples-operator-tls" not found
Jul 16 21:49:32.215540 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215396    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 21:49:32.215540 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215406    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758897b654-n8j2k: secret "image-registry-tls" not found
Jul 16 21:49:32.215540 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.215434    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls podName:84797633-98b5-4ce4-90ac-779d24813acf nodeName:}" failed. No retries permitted until 2026-07-16 21:49:34.21542437 +0000 UTC m=+40.604698137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls") pod "image-registry-758897b654-n8j2k" (UID: "84797633-98b5-4ce4-90ac-779d24813acf") : secret "image-registry-tls" not found
Jul 16 21:49:32.247135 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.246382    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:32.316610 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.315387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:32.316610 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.315543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:32.316610 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.315686    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 21:49:32.316610 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.315748    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert podName:88805df0-8f06-432d-9316-c97536c8bc08 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:34.315729522 +0000 UTC m=+40.705003289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-nj2nn" (UID: "88805df0-8f06-432d-9316-c97536c8bc08") : secret "networking-console-plugin-cert" not found
Jul 16 21:49:32.316610 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.316151    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:49:32.316610 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:32.316201    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls podName:2e635ee7-8ae6-4a90-a66f-e33cfd434838 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:34.316185968 +0000 UTC m=+40.705459738 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls") pod "dns-default-wn4d2" (UID: "2e635ee7-8ae6-4a90-a66f-e33cfd434838") : secret "dns-default-metrics-tls" not found
Jul 16 21:49:32.355613 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.355367    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb" event={"ID":"98296b74-f3c7-4a28-aba5-fd0bfab5d32f","Type":"ContainerStarted","Data":"3dcc36a659bf16362786f4d87f3713c493744838c6d051975f6fdea4c2c818b6"}
Jul 16 21:49:32.359555 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.359522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-77j96" event={"ID":"cb9b083e-7066-40e4-bce1-366de4032eaa","Type":"ContainerStarted","Data":"ed2f6a00f973552193745827be9cfb8970e1cab3fb8948431b42741ddfb01dcb"}
Jul 16 21:49:32.365831 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.364679    2564 generic.go:356] "Generic (PLEG): container finished" podID="eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e" containerID="bd79fdcdbdfebbc209e8f8a1da25d70fa3701155e7ad4dd8844257a485ec83d7" exitCode=0
Jul 16 21:49:32.365831 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.364757    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerDied","Data":"bd79fdcdbdfebbc209e8f8a1da25d70fa3701155e7ad4dd8844257a485ec83d7"}
Jul 16 21:49:32.374021 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.373690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8pnfs" event={"ID":"3b323658-e5c3-4845-9038-a8695bfed36f","Type":"ContainerStarted","Data":"2345946374fe4c30efacce62cd203b3a8d016dddd11f35529eaea4cdea589986"}
Jul 16 21:49:32.379104 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.379040    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b" event={"ID":"52af9bbb-a233-445a-98e6-22f0cc478c60","Type":"ContainerStarted","Data":"2daa51a1910c09cb7fb01262362a0c60c3e1e649d1e76fff09f8320e8361b0c7"}
Jul 16 21:49:32.382501 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.382415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-trdvm" event={"ID":"1173fb03-41c1-4aad-902f-7b7f6d9bd371","Type":"ContainerStarted","Data":"4d2bf087f618c07602ff0688a8f4afb3dc7a2a44c7b052bc9559a80390f34343"}
Jul 16 21:49:32.388452 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.388384    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk" event={"ID":"512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae","Type":"ContainerStarted","Data":"37d688a2302e88875c6e9fc75e94f4c30fc41fe2b3d41b3bb3d875a358900933"}
Jul 16 21:49:32.640056 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.639968    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:32.703093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.702109    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"]
Jul 16 21:49:32.705598 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.705224    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.716429 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.716402    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 16 21:49:32.716905 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.716883    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 16 21:49:32.718227 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.717212    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 16 21:49:32.718227 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.717460    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 16 21:49:32.731957 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.731402    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"]
Jul 16 21:49:32.736413 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.736383    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"]
Jul 16 21:49:32.737454 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.736626    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:32.740775 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.740757    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:32.741564 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.741275    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"]
Jul 16 21:49:32.750182 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.750161    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6hh2n\""
Jul 16 21:49:32.753777 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.753753    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"]
Jul 16 21:49:32.754368 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.754320    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 16 21:49:32.754658 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.754639    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 16 21:49:32.755560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.755102    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 16 21:49:32.755560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.755326    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 16 21:49:32.755560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.755524    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 16 21:49:32.776986 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.776743    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"]
Jul 16 21:49:32.822314 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.821932    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd5c2023-31e7-47f8-8d56-126fadbcc777-tmp\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.822314 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.822066    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fd5c2023-31e7-47f8-8d56-126fadbcc777-klusterlet-config\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.822314 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.822100    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb5kh\" (UniqueName: \"kubernetes.io/projected/fd5c2023-31e7-47f8-8d56-126fadbcc777-kube-api-access-mb5kh\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-hub\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923579    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923713    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-ca\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd5c2023-31e7-47f8-8d56-126fadbcc777-tmp\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hbk4\" (UniqueName: \"kubernetes.io/projected/85ae485f-b301-4f20-8fed-e379d1beaa90-kube-api-access-7hbk4\") pod \"managed-serviceaccount-addon-agent-74bb865b48-t2mbd\" (UID: \"85ae485f-b301-4f20-8fed-e379d1beaa90\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923899    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxlcj\" (UniqueName: \"kubernetes.io/projected/3af01737-a87a-493c-8b1e-955d4d51d03a-kube-api-access-hxlcj\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.923982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fd5c2023-31e7-47f8-8d56-126fadbcc777-klusterlet-config\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.924011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb5kh\" (UniqueName: \"kubernetes.io/projected/fd5c2023-31e7-47f8-8d56-126fadbcc777-kube-api-access-mb5kh\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.924037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/85ae485f-b301-4f20-8fed-e379d1beaa90-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bb865b48-t2mbd\" (UID: \"85ae485f-b301-4f20-8fed-e379d1beaa90\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.924083    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af01737-a87a-493c-8b1e-955d4d51d03a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:32.925326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.924585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd5c2023-31e7-47f8-8d56-126fadbcc777-tmp\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.938931 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.938858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fd5c2023-31e7-47f8-8d56-126fadbcc777-klusterlet-config\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:32.957609 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.957556    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:32.967414 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:32.961194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb5kh\" (UniqueName: \"kubernetes.io/projected/fd5c2023-31e7-47f8-8d56-126fadbcc777-kube-api-access-mb5kh\") pod \"klusterlet-addon-workmgr-858556c657-rsdcf\" (UID: \"fd5c2023-31e7-47f8-8d56-126fadbcc777\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:33.019420 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.019381    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:33.025306 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.025471 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-ca\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.025471 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025401    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hbk4\" (UniqueName: \"kubernetes.io/projected/85ae485f-b301-4f20-8fed-e379d1beaa90-kube-api-access-7hbk4\") pod \"managed-serviceaccount-addon-agent-74bb865b48-t2mbd\" (UID: \"85ae485f-b301-4f20-8fed-e379d1beaa90\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:33.025471 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025443    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxlcj\" (UniqueName: \"kubernetes.io/projected/3af01737-a87a-493c-8b1e-955d4d51d03a-kube-api-access-hxlcj\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.025695 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/85ae485f-b301-4f20-8fed-e379d1beaa90-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bb865b48-t2mbd\" (UID: \"85ae485f-b301-4f20-8fed-e379d1beaa90\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:33.025695 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af01737-a87a-493c-8b1e-955d4d51d03a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.025695 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025617    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-hub\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.025695 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.025640    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.027735 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.027273    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af01737-a87a-493c-8b1e-955d4d51d03a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.033997 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.033971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-hub\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.039026 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.038980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.041026 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.040993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/85ae485f-b301-4f20-8fed-e379d1beaa90-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bb865b48-t2mbd\" (UID: \"85ae485f-b301-4f20-8fed-e379d1beaa90\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:33.046093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.046070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-ca\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.046618 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.046596    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3af01737-a87a-493c-8b1e-955d4d51d03a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.046844 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.046824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hbk4\" (UniqueName: \"kubernetes.io/projected/85ae485f-b301-4f20-8fed-e379d1beaa90-kube-api-access-7hbk4\") pod \"managed-serviceaccount-addon-agent-74bb865b48-t2mbd\" (UID: \"85ae485f-b301-4f20-8fed-e379d1beaa90\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:33.057975 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.057924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxlcj\" (UniqueName: \"kubernetes.io/projected/3af01737-a87a-493c-8b1e-955d4d51d03a-kube-api-access-hxlcj\") pod \"cluster-proxy-proxy-agent-6766cb4db4-v89lk\" (UID: \"3af01737-a87a-493c-8b1e-955d4d51d03a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.062520 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.062496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"
Jul 16 21:49:33.074006 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.073865    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:49:33.408225 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.405385    2564 generic.go:356] "Generic (PLEG): container finished" podID="eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e" containerID="e7e4dfafd5f9412992ef59768d0bffdf3864e8c9d4f284c6452f87cecc476800" exitCode=0
Jul 16 21:49:33.408225 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.405477    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerDied","Data":"e7e4dfafd5f9412992ef59768d0bffdf3864e8c9d4f284c6452f87cecc476800"}
Jul 16 21:49:33.523599 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.523357    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"]
Jul 16 21:49:33.544463 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.544437    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"]
Jul 16 21:49:33.553697 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:33.552395    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd"]
Jul 16 21:49:33.688619 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:33.688583    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85ae485f_b301_4f20_8fed_e379d1beaa90.slice/crio-c99d0bd312139f15ad3b6a46ebd6c95cea516a805f0efdea7388b76023931665 WatchSource:0}: Error finding container c99d0bd312139f15ad3b6a46ebd6c95cea516a805f0efdea7388b76023931665: Status 404 returned error can't find the container with id c99d0bd312139f15ad3b6a46ebd6c95cea516a805f0efdea7388b76023931665
Jul 16 21:49:33.689929 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:33.689885    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5c2023_31e7_47f8_8d56_126fadbcc777.slice/crio-7d387f8bf1cc66c286fdba908e352aa9b9709fae5bf439c7811b92d0e1145ffe WatchSource:0}: Error finding container 7d387f8bf1cc66c286fdba908e352aa9b9709fae5bf439c7811b92d0e1145ffe: Status 404 returned error can't find the container with id 7d387f8bf1cc66c286fdba908e352aa9b9709fae5bf439c7811b92d0e1145ffe
Jul 16 21:49:33.694712 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:33.694684    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3af01737_a87a_493c_8b1e_955d4d51d03a.slice/crio-c5e277bb56669fc64da389386ddaa8b28c410305405f5c583c44d34561df0436 WatchSource:0}: Error finding container c5e277bb56669fc64da389386ddaa8b28c410305405f5c583c44d34561df0436: Status 404 returned error can't find the container with id c5e277bb56669fc64da389386ddaa8b28c410305405f5c583c44d34561df0436
Jul 16 21:49:34.241388 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.241304    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:34.241545 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.241434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:34.241545 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241478    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:38.241459917 +0000 UTC m=+44.630733680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : configmap references non-existent config key: service-ca.crt
Jul 16 21:49:34.241668 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241547    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:49:34.241668 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.241564    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:34.241668 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241603    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:38.241586607 +0000 UTC m=+44.630860407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : secret "router-metrics-certs-default" not found
Jul 16 21:49:34.241668 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241655    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241691    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls podName:f6340ec6-1b89-4f62-a863-22358fc973ea nodeName:}" failed. No retries permitted until 2026-07-16 21:49:38.241679526 +0000 UTC m=+44.630953289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2f7rl" (UID: "f6340ec6-1b89-4f62-a863-22358fc973ea") : secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.241651    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241714    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.241740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241750    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls podName:b61aa2f1-c376-4a76-a60b-0ba45afdbd66 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:38.241738676 +0000 UTC m=+44.631012442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8dg4p" (UID: "b61aa2f1-c376-4a76-a60b-0ba45afdbd66") : secret "samples-operator-tls" not found
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241793    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241803    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758897b654-n8j2k: secret "image-registry-tls" not found
Jul 16 21:49:34.241858 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.241831    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls podName:84797633-98b5-4ce4-90ac-779d24813acf nodeName:}" failed. No retries permitted until 2026-07-16 21:49:38.241823251 +0000 UTC m=+44.631097015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls") pod "image-registry-758897b654-n8j2k" (UID: "84797633-98b5-4ce4-90ac-779d24813acf") : secret "image-registry-tls" not found
Jul 16 21:49:34.342872 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.342836    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:34.343063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.342964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:34.343063 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.343024    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 21:49:34.343224 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.343101    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:49:34.343224 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.343158    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls podName:2e635ee7-8ae6-4a90-a66f-e33cfd434838 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:38.343136697 +0000 UTC m=+44.732410464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls") pod "dns-default-wn4d2" (UID: "2e635ee7-8ae6-4a90-a66f-e33cfd434838") : secret "dns-default-metrics-tls" not found
Jul 16 21:49:34.343224 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:34.343182    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert podName:88805df0-8f06-432d-9316-c97536c8bc08 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:38.343171024 +0000 UTC m=+44.732444793 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-nj2nn" (UID: "88805df0-8f06-432d-9316-c97536c8bc08") : secret "networking-console-plugin-cert" not found
Jul 16 21:49:34.414357 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.414309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-czz97" event={"ID":"eec3f9ce-b783-4b72-a0ea-7ea8bc02e52e","Type":"ContainerStarted","Data":"ab9ed7f75eb5cd7c91dc69783ff7dd15c60cc45a5957cdb243a97c26b3ee6293"}
Jul 16 21:49:34.415830 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.415791    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd" event={"ID":"85ae485f-b301-4f20-8fed-e379d1beaa90","Type":"ContainerStarted","Data":"c99d0bd312139f15ad3b6a46ebd6c95cea516a805f0efdea7388b76023931665"}
Jul 16 21:49:34.417356 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.417328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf" event={"ID":"fd5c2023-31e7-47f8-8d56-126fadbcc777","Type":"ContainerStarted","Data":"7d387f8bf1cc66c286fdba908e352aa9b9709fae5bf439c7811b92d0e1145ffe"}
Jul 16 21:49:34.418658 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.418631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" event={"ID":"3af01737-a87a-493c-8b1e-955d4d51d03a","Type":"ContainerStarted","Data":"c5e277bb56669fc64da389386ddaa8b28c410305405f5c583c44d34561df0436"}
Jul 16 21:49:34.455376 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.455313    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-czz97" podStartSLOduration=7.497127213 podStartE2EDuration="39.455291442s" totalImagesPullingTime="31.958164229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:34.452426515 +0000 UTC m=+40.841700346" watchObservedRunningTime="2026-07-16 21:49:34.455291442 +0000 UTC m=+40.844565228"
Jul 16 21:49:34.653572 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.653421    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:34.763586 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.763415    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:34.874493 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:34.874124    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:35.790218 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:35.790178    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:35.884543 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:35.884508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:37.174292 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:37.174251    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:37.274362 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:37.274331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:38.277869    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:38.277947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:38.278010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278031    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:38.278053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:38.278092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278117    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:46.278096896 +0000 UTC m=+52.667370666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : secret "router-metrics-certs-default" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278199    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:46.278182263 +0000 UTC m=+52.667456041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : configmap references non-existent config key: service-ca.crt
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278319    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278356    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls podName:f6340ec6-1b89-4f62-a863-22358fc973ea nodeName:}" failed. No retries permitted until 2026-07-16 21:49:46.278347942 +0000 UTC m=+52.667621705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2f7rl" (UID: "f6340ec6-1b89-4f62-a863-22358fc973ea") : secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278411    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278448    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls podName:b61aa2f1-c376-4a76-a60b-0ba45afdbd66 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:46.278436768 +0000 UTC m=+52.667710541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8dg4p" (UID: "b61aa2f1-c376-4a76-a60b-0ba45afdbd66") : secret "samples-operator-tls" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278515    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278526    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758897b654-n8j2k: secret "image-registry-tls" not found
Jul 16 21:49:38.278584 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.278563    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls podName:84797633-98b5-4ce4-90ac-779d24813acf nodeName:}" failed. No retries permitted until 2026-07-16 21:49:46.278552159 +0000 UTC m=+52.667825924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls") pod "image-registry-758897b654-n8j2k" (UID: "84797633-98b5-4ce4-90ac-779d24813acf") : secret "image-registry-tls" not found
Jul 16 21:49:38.379455 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:38.379414    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:38.379647 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:38.379515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:38.379647 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.379578    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 21:49:38.379647 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.379627    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:49:38.379801 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.379653    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert podName:88805df0-8f06-432d-9316-c97536c8bc08 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:46.379635676 +0000 UTC m=+52.768909443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-nj2nn" (UID: "88805df0-8f06-432d-9316-c97536c8bc08") : secret "networking-console-plugin-cert" not found
Jul 16 21:49:38.379801 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:38.379678    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls podName:2e635ee7-8ae6-4a90-a66f-e33cfd434838 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:46.379666055 +0000 UTC m=+52.768939819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls") pod "dns-default-wn4d2" (UID: "2e635ee7-8ae6-4a90-a66f-e33cfd434838") : secret "dns-default-metrics-tls" not found
Jul 16 21:49:44.937851 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:44.937785    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:45.450712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.450680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8pnfs" event={"ID":"3b323658-e5c3-4845-9038-a8695bfed36f","Type":"ContainerStarted","Data":"68f69f7922b13bdfc151a9b8ea3b3f327c12c499767c9b93727d490f9d5273a8"}
Jul 16 21:49:45.452388 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.452356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b" event={"ID":"52af9bbb-a233-445a-98e6-22f0cc478c60","Type":"ContainerStarted","Data":"4818d9b124c42c76809eae4c8cf08bf9baf7a9946a69c244e2383811efb6eb3f"}
Jul 16 21:49:45.453884 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.453858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd" event={"ID":"85ae485f-b301-4f20-8fed-e379d1beaa90","Type":"ContainerStarted","Data":"36aadf08cd87c2e58ed5332cc5ba553e0ed3a75fe756b930a61222bd7ef67bb5"}
Jul 16 21:49:45.455395 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.455353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4" event={"ID":"748c76b0-d147-4d51-9377-9409e9faefa9","Type":"ContainerStarted","Data":"3d6519d3a10bef6f9c81c6f61e908dec6ab988e10cccd3c891f4574309f31c61"}
Jul 16 21:49:45.456974 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.456946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-trdvm" event={"ID":"1173fb03-41c1-4aad-902f-7b7f6d9bd371","Type":"ContainerStarted","Data":"ca83155c565ebfdecd6e214d74d0e00f5c87ccd12caf70396214c11a9ac7eaf3"}
Jul 16 21:49:45.458406 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.458382    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf" event={"ID":"fd5c2023-31e7-47f8-8d56-126fadbcc777","Type":"ContainerStarted","Data":"f4bdbe971a6a60cd65620e882155d8172220bfe34e91210795911c6cdde8a732"}
Jul 16 21:49:45.458519 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.458502    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:45.460124 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.460097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk" event={"ID":"512bb7d4-6c82-4e95-aa19-4f0c2f7f4bae","Type":"ContainerStarted","Data":"35d25ce996525214aa0ce3b18d6a035f16902c25000e1e910f1390ec98533a85"}
Jul 16 21:49:45.460653 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.460635    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf"
Jul 16 21:49:45.461795 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.461772    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb" event={"ID":"98296b74-f3c7-4a28-aba5-fd0bfab5d32f","Type":"ContainerStarted","Data":"680722bfc728106d18b1c44aa9fe6ee7e42f202253e45b99c25676c43b57a586"}
Jul 16 21:49:45.463924 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.463885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" event={"ID":"3af01737-a87a-493c-8b1e-955d4d51d03a","Type":"ContainerStarted","Data":"b33d8388e296ed2d6c3d4690a7539b209f5b1dd21fa4ad666fdd8cfb9df1d88b"}
Jul 16 21:49:45.464029 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.463959    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" event={"ID":"3af01737-a87a-493c-8b1e-955d4d51d03a","Type":"ContainerStarted","Data":"63bd6fd92ec67c31ce10fcdfe89f9a57389af00d5d1b68848f7abe8974a2f36d"}
Jul 16 21:49:45.464029 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.463993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" event={"ID":"3af01737-a87a-493c-8b1e-955d4d51d03a","Type":"ContainerStarted","Data":"5932f733ecc4bd7c68c99663b2ba114b04b49d7cc782bb2fcb0b6fdfa0331344"}
Jul 16 21:49:45.465680 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.465662    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/0.log"
Jul 16 21:49:45.465772 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.465701    2564 generic.go:356] "Generic (PLEG): container finished" podID="cb9b083e-7066-40e4-bce1-366de4032eaa" containerID="4c7813caddb9ded6028f7e5ce4cedbaa221e1db6957fb5b820c7af5c16feac14" exitCode=255
Jul 16 21:49:45.465772 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.465735    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-77j96" event={"ID":"cb9b083e-7066-40e4-bce1-366de4032eaa","Type":"ContainerDied","Data":"4c7813caddb9ded6028f7e5ce4cedbaa221e1db6957fb5b820c7af5c16feac14"}
Jul 16 21:49:45.465938 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.465903    2564 scope.go:117] "RemoveContainer" containerID="4c7813caddb9ded6028f7e5ce4cedbaa221e1db6957fb5b820c7af5c16feac14"
Jul 16 21:49:45.468647 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.468574    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8pnfs" podStartSLOduration=2.42229011 podStartE2EDuration="15.468558355s" totalImagesPullingTime="13.046268245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.467501938 +0000 UTC m=+51.856775723" watchObservedRunningTime="2026-07-16 21:49:45.468558355 +0000 UTC m=+51.857832143"
Jul 16 21:49:45.487325 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.487273    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-trdvm" podStartSLOduration=36.374221187 podStartE2EDuration="49.487255269s" totalImagesPullingTime="13.113034082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.48612284 +0000 UTC m=+51.875396625" watchObservedRunningTime="2026-07-16 21:49:45.487255269 +0000 UTC m=+51.876529058"
Jul 16 21:49:45.507275 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.507235    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-858556c657-rsdcf" podStartSLOduration=2.704499935 podStartE2EDuration="13.507219682s" totalImagesPullingTime="10.802719747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.50585753 +0000 UTC m=+51.895131318" watchObservedRunningTime="2026-07-16 21:49:45.507219682 +0000 UTC m=+51.896493462"
Jul 16 21:49:45.529634 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.529589    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" podStartSLOduration=2.703015893 podStartE2EDuration="13.529573421s" totalImagesPullingTime="10.826557528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.527477166 +0000 UTC m=+51.916750953" watchObservedRunningTime="2026-07-16 21:49:45.529573421 +0000 UTC m=+51.918847201"
Jul 16 21:49:45.566245 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.566201    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pwrkk" podStartSLOduration=34.423535704 podStartE2EDuration="47.566183732s" totalImagesPullingTime="13.142648028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.546427589 +0000 UTC m=+51.935701377" watchObservedRunningTime="2026-07-16 21:49:45.566183732 +0000 UTC m=+51.955457498"
Jul 16 21:49:45.591991 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.591939    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4" podStartSLOduration=36.479389863 podStartE2EDuration="49.591896201s" totalImagesPullingTime="13.112506338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.589415134 +0000 UTC m=+51.978688921" watchObservedRunningTime="2026-07-16 21:49:45.591896201 +0000 UTC m=+51.981169988"
Jul 16 21:49:45.615928 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.615340    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-r4frb" podStartSLOduration=44.330187193 podStartE2EDuration="49.615318175s" totalImagesPullingTime="5.285130982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.613290965 +0000 UTC m=+52.002564754" watchObservedRunningTime="2026-07-16 21:49:45.615318175 +0000 UTC m=+52.004591963"
Jul 16 21:49:45.631927 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.631416    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bb865b48-t2mbd" podStartSLOduration=2.819749601 podStartE2EDuration="13.631395981s" totalImagesPullingTime="10.81164638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.629203344 +0000 UTC m=+52.018477131" watchObservedRunningTime="2026-07-16 21:49:45.631395981 +0000 UTC m=+52.020669770"
Jul 16 21:49:45.655243 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.655202    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b" podStartSLOduration=36.553588367 podStartE2EDuration="49.655186901s" totalImagesPullingTime="13.101598534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:45.65441159 +0000 UTC m=+52.043685377" watchObservedRunningTime="2026-07-16 21:49:45.655186901 +0000 UTC m=+52.044460684"
Jul 16 21:49:45.730553 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.730489    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:45.792212 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.792177    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:45.884663 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.884589    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:45.984094 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:45.984019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:46.085432 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.085395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:46.281965 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.281878    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:46.354798 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.354766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:46.354970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.354814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:49:46.354970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.354863    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:49:46.354970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.354903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:49:46.354970 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.354955    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355006    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355017    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355025    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:02.355004152 +0000 UTC m=+68.744277934 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : secret "router-metrics-certs-default" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355029    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758897b654-n8j2k: secret "image-registry-tls" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355055    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355073    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls podName:84797633-98b5-4ce4-90ac-779d24813acf nodeName:}" failed. No retries permitted until 2026-07-16 21:50:02.355058209 +0000 UTC m=+68.744331978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls") pod "image-registry-758897b654-n8j2k" (UID: "84797633-98b5-4ce4-90ac-779d24813acf") : secret "image-registry-tls" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.355066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355094    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls podName:f6340ec6-1b89-4f62-a863-22358fc973ea nodeName:}" failed. No retries permitted until 2026-07-16 21:50:02.355080978 +0000 UTC m=+68.744354745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-2f7rl" (UID: "f6340ec6-1b89-4f62-a863-22358fc973ea") : secret "cluster-monitoring-operator-tls" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355123    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls podName:b61aa2f1-c376-4a76-a60b-0ba45afdbd66 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:02.35511151 +0000 UTC m=+68.744385278 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8dg4p" (UID: "b61aa2f1-c376-4a76-a60b-0ba45afdbd66") : secret "samples-operator-tls" not found
Jul 16 21:49:46.355177 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.355138    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle podName:d70792ad-fa1c-4f45-be6c-05892fd3fc19 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:02.355131878 +0000 UTC m=+68.744405642 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle") pod "router-default-64db4fc64d-wlpxp" (UID: "d70792ad-fa1c-4f45-be6c-05892fd3fc19") : configmap references non-existent config key: service-ca.crt
Jul 16 21:49:46.381943 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.381900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:46.456057 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.456019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:49:46.456214 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.456152    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 21:49:46.456214 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.456166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:49:46.456214 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.456213    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls podName:2e635ee7-8ae6-4a90-a66f-e33cfd434838 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:02.456197989 +0000 UTC m=+68.845471774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls") pod "dns-default-wn4d2" (UID: "2e635ee7-8ae6-4a90-a66f-e33cfd434838") : secret "dns-default-metrics-tls" not found
Jul 16 21:49:46.456386 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.456270    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 21:49:46.456386 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.456328    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert podName:88805df0-8f06-432d-9316-c97536c8bc08 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:02.456311043 +0000 UTC m=+68.845584822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-nj2nn" (UID: "88805df0-8f06-432d-9316-c97536c8bc08") : secret "networking-console-plugin-cert" not found
Jul 16 21:49:46.471504 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.471483    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 21:49:46.471889 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.471873    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/0.log"
Jul 16 21:49:46.471982 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.471940    2564 generic.go:356] "Generic (PLEG): container finished" podID="cb9b083e-7066-40e4-bce1-366de4032eaa" containerID="8b1e5e6aab2bf1fb2de16d9ad2b1944e993264f0296b2916f3a64a9d6ba074a0" exitCode=255
Jul 16 21:49:46.472089 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.472021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-77j96" event={"ID":"cb9b083e-7066-40e4-bce1-366de4032eaa","Type":"ContainerDied","Data":"8b1e5e6aab2bf1fb2de16d9ad2b1944e993264f0296b2916f3a64a9d6ba074a0"}
Jul 16 21:49:46.472149 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.472064    2564 scope.go:117] "RemoveContainer" containerID="4c7813caddb9ded6028f7e5ce4cedbaa221e1db6957fb5b820c7af5c16feac14"
Jul 16 21:49:46.472314 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.472260    2564 scope.go:117] "RemoveContainer" containerID="8b1e5e6aab2bf1fb2de16d9ad2b1944e993264f0296b2916f3a64a9d6ba074a0"
Jul 16 21:49:46.472457 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:46.472436    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-77j96_openshift-console-operator(cb9b083e-7066-40e4-bce1-366de4032eaa)\"" pod="openshift-console-operator/console-operator-565b7b569d-77j96" podUID="cb9b083e-7066-40e4-bce1-366de4032eaa"
Jul 16 21:49:46.537603 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.537526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:46.643053 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:46.643015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:47.097082 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.097048    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:47.284973 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.284936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:47.345690 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.345651    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9dtrc"]
Jul 16 21:49:47.381468 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.381396    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9dtrc"]
Jul 16 21:49:47.381613 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.381527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.384024 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.384000    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 21:49:47.384154 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.384040    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vv8qm\""
Jul 16 21:49:47.384855 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.384836    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 21:49:47.425275 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.425247    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:47.464800 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.464769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-data-volume\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.464959 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.464808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.464959 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.464885    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-crio-socket\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.464959 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.464940    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clkzs\" (UniqueName: \"kubernetes.io/projected/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-api-access-clkzs\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.465094 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.464978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.477413 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.477393    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 21:49:47.477700 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.477686    2564 scope.go:117] "RemoveContainer" containerID="8b1e5e6aab2bf1fb2de16d9ad2b1944e993264f0296b2916f3a64a9d6ba074a0"
Jul 16 21:49:47.477847 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:47.477829    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-77j96_openshift-console-operator(cb9b083e-7066-40e4-bce1-366de4032eaa)\"" pod="openshift-console-operator/console-operator-565b7b569d-77j96" podUID="cb9b083e-7066-40e4-bce1-366de4032eaa"
Jul 16 21:49:47.565789 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.565758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-crio-socket\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.565973 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.565816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clkzs\" (UniqueName: \"kubernetes.io/projected/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-api-access-clkzs\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.565973 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.565870    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.566102 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.566064    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-crio-socket\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.566225 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.566197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-data-volume\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.566330 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.566270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.566491 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:47.566473    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:47.566564 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:47.566535    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:48.066516383 +0000 UTC m=+54.455790165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:47.566564 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:47.566558    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:49:47.566677 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:47.566606    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:48.066588272 +0000 UTC m=+54.455862042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : configmap "kube-rbac-proxy" not found
Jul 16 21:49:47.566999 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.566975    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-data-volume\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:47.576300 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:47.576282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clkzs\" (UniqueName: \"kubernetes.io/projected/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-api-access-clkzs\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:48.070839 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:48.070800    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:48.071036 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:48.070934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:48.071036 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:48.070947    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:49:48.071036 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:48.071007    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:49.070991767 +0000 UTC m=+55.460265530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : configmap "kube-rbac-proxy" not found
Jul 16 21:49:48.071188 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:48.071061    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:48.071188 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:48.071122    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:49.071107421 +0000 UTC m=+55.460381185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:48.656503 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:48.656471    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:48.729402 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:48.729364    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:49.080927 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:49.080890    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:49.081104 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:49.080997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:49.081104 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:49.081011    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:49:49.081104 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:49.081068    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:51.081051938 +0000 UTC m=+57.470325703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : configmap "kube-rbac-proxy" not found
Jul 16 21:49:49.081104 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:49.081089    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:49.081234 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:49.081129    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:51.081117026 +0000 UTC m=+57.470390790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:49.692461 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:49.692428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:50.347302 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:50.347277    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s8d4d_6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41/dns-node-resolver/0.log"
Jul 16 21:49:50.821057 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:50.821025    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:50.821409 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:50.821075    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:49:50.821409 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:50.821389    2564 scope.go:117] "RemoveContainer" containerID="8b1e5e6aab2bf1fb2de16d9ad2b1944e993264f0296b2916f3a64a9d6ba074a0"
Jul 16 21:49:50.821552 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:50.821534    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-77j96_openshift-console-operator(cb9b083e-7066-40e4-bce1-366de4032eaa)\"" pod="openshift-console-operator/console-operator-565b7b569d-77j96" podUID="cb9b083e-7066-40e4-bce1-366de4032eaa"
Jul 16 21:49:51.098080 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:51.097998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:51.098216 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:51.098095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:51.098216 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:51.098138    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:51.098216 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:51.098200    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:49:51.098308 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:51.098206    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:55.098190271 +0000 UTC m=+61.487464036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:51.098308 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:51.098254    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:55.098241628 +0000 UTC m=+61.487515392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : configmap "kube-rbac-proxy" not found
Jul 16 21:49:51.144311 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:51.144290    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nwfwh_e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e/node-ca/0.log"
Jul 16 21:49:51.744420 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:51.744388    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8pnfs_3b323658-e5c3-4845-9038-a8695bfed36f/serve-healthcheck-canary/0.log"
Jul 16 21:49:52.751948 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:52.751903    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-qxdl4_748c76b0-d147-4d51-9377-9409e9faefa9/kube-storage-version-migrator-operator/0.log"
Jul 16 21:49:53.252430 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:53.252393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:53.389967 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:53.389898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:53.493399 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:53.493340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:53.609971 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:53.609848    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:53.737965 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:53.737898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:54.538997 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:54.538957    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:54.939507 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:54.939481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:55.135758 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:55.135726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:55.135961 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:55.135815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:49:55.135961 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:55.135848    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 21:49:55.135961 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:55.135932    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:55.135961 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:55.135937    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:03.135898503 +0000 UTC m=+69.525172267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : configmap "kube-rbac-proxy" not found
Jul 16 21:49:55.136121 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:55.135976    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls podName:38f4b3d2-9161-45a6-ad8c-cc0c509d65a4 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:03.135964666 +0000 UTC m=+69.525238430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9dtrc" (UID: "38f4b3d2-9161-45a6-ad8c-cc0c509d65a4") : secret "insights-runtime-extractor-tls" not found
Jul 16 21:49:55.145542 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:55.145508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:55.341012 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:55.340948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tm8hd"
Jul 16 21:49:55.400975 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:55.400936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:55.730194 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:55.730166    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:56.140285 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:56.140209    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:56.239434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:56.239391    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:56.756029 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:56.755997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:57.123311 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:57.123223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:57.226565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:57.226537    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:57.582892 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:57.582862    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:57.737703 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:57.737669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:57.803351 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:57.803313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:58.382964 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.382930    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:58.549992 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.549955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:58.767364 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.767334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:58.767512 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.767455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:49:58.770689 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.770671    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:49:58.770689 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.770683    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:49:58.777685 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:58.777665    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 16 21:49:58.777780 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:49:58.777735    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs podName:9d31ef25-6751-41d8-9583-11a3455909e6 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:02.777714139 +0000 UTC m=+129.166987905 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs") pod "network-metrics-daemon-hzwzf" (UID: "9d31ef25-6751-41d8-9583-11a3455909e6") : secret "metrics-daemon-secret" not found
Jul 16 21:49:58.779841 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.779821    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9adec6-0471-441e-8c76-2dbdd18bfc18-original-pull-secret\") pod \"global-pull-secret-syncer-zkdf9\" (UID: \"5f9adec6-0471-441e-8c76-2dbdd18bfc18\") " pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:58.868505 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.868475    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zkdf9"
Jul 16 21:49:58.939694 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.939650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:58.969456 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.969429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:58.971894 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.971867    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlr2l\" (UniqueName: \"kubernetes.io/projected/4cbd2da1-d5ea-41f9-a004-dd503966a636-kube-api-access-tlr2l\") pod \"network-check-target-8ltd5\" (UID: \"4cbd2da1-d5ea-41f9-a004-dd503966a636\") " pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:58.996039 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:58.996015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zkdf9"]
Jul 16 21:49:58.998784 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:58.998762    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9adec6_0471_441e_8c76_2dbdd18bfc18.slice/crio-d704f0602ca216dd5840d3c2bb72d0e651ee0c96b6b2beb1decd5bd4b052a6e6 WatchSource:0}: Error finding container d704f0602ca216dd5840d3c2bb72d0e651ee0c96b6b2beb1decd5bd4b052a6e6: Status 404 returned error can't find the container with id d704f0602ca216dd5840d3c2bb72d0e651ee0c96b6b2beb1decd5bd4b052a6e6
Jul 16 21:49:59.159782 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.159707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xjnvp\""
Jul 16 21:49:59.167889 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.167871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:59.293942 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.293902    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ltd5"]
Jul 16 21:49:59.296216 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:49:59.296192    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbd2da1_d5ea_41f9_a004_dd503966a636.slice/crio-bff59e6726a1b7cde2a588998925b470a0dcb41143630db4596748d68f1f40a9 WatchSource:0}: Error finding container bff59e6726a1b7cde2a588998925b470a0dcb41143630db4596748d68f1f40a9: Status 404 returned error can't find the container with id bff59e6726a1b7cde2a588998925b470a0dcb41143630db4596748d68f1f40a9
Jul 16 21:49:59.518092 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.518053    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zkdf9" event={"ID":"5f9adec6-0471-441e-8c76-2dbdd18bfc18","Type":"ContainerStarted","Data":"d704f0602ca216dd5840d3c2bb72d0e651ee0c96b6b2beb1decd5bd4b052a6e6"}
Jul 16 21:49:59.519447 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.519424    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ltd5" event={"ID":"4cbd2da1-d5ea-41f9-a004-dd503966a636","Type":"ContainerStarted","Data":"ad4fc1c63ffca491dead47beddefebb7e97911a6a4bcdced9316d86692d57ebf"}
Jul 16 21:49:59.519579 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.519465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ltd5" event={"ID":"4cbd2da1-d5ea-41f9-a004-dd503966a636","Type":"ContainerStarted","Data":"bff59e6726a1b7cde2a588998925b470a0dcb41143630db4596748d68f1f40a9"}
Jul 16 21:49:59.519579 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.519559    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:49:59.537686 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.537641    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8ltd5" podStartSLOduration=64.537625978 podStartE2EDuration="1m4.537625978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:59.537320963 +0000 UTC m=+65.926594750" watchObservedRunningTime="2026-07-16 21:49:59.537625978 +0000 UTC m=+65.926899768"
Jul 16 21:49:59.543865 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.543838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:59.743282 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.743241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:49:59.943875 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:49:59.943838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:00.339543 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:00.339449    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:00.532415 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:00.532376    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:00.630039 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:00.629821    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:01.120206 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.120177    2564 scope.go:117] "RemoveContainer" containerID="8b1e5e6aab2bf1fb2de16d9ad2b1944e993264f0296b2916f3a64a9d6ba074a0"
Jul 16 21:50:01.182469 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.182441    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:01.279079 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.279045    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:01.529944 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.529258    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 21:50:01.529944 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.529356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-77j96" event={"ID":"cb9b083e-7066-40e4-bce1-366de4032eaa","Type":"ContainerStarted","Data":"e172196a194bd6930cae9df90a8282e8476d29ce83952f14c51c8baceabb4db5"}
Jul 16 21:50:01.529944 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.529693    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:50:01.764250 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.764220    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:01.876674 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.876604    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-77j96"
Jul 16 21:50:01.896520 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.896469    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-77j96" podStartSLOduration=52.758403799999996 podStartE2EDuration="1m5.896448784s" totalImagesPullingTime="13.138044984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:01.558889468 +0000 UTC m=+67.948163254" watchObservedRunningTime="2026-07-16 21:50:01.896448784 +0000 UTC m=+68.285722570"
Jul 16 21:50:01.946303 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:01.946270    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:02.164842 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.164731    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:02.251182 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.251143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:02.343895 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.343859    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:02.403570 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.403538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:02.403754 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.403588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:50:02.403883 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.403854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:50:02.403977 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.403933    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:50:02.404051 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.403976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:02.404994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.404885    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70792ad-fa1c-4f45-be6c-05892fd3fc19-service-ca-bundle\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:02.406557 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.406531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70792ad-fa1c-4f45-be6c-05892fd3fc19-metrics-certs\") pod \"router-default-64db4fc64d-wlpxp\" (UID: \"d70792ad-fa1c-4f45-be6c-05892fd3fc19\") " pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:02.406682 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.406536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f6340ec6-1b89-4f62-a863-22358fc973ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-2f7rl\" (UID: \"f6340ec6-1b89-4f62-a863-22358fc973ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:50:02.406747 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.406720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b61aa2f1-c376-4a76-a60b-0ba45afdbd66-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8dg4p\" (UID: \"b61aa2f1-c376-4a76-a60b-0ba45afdbd66\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:50:02.407227 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.407203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"image-registry-758897b654-n8j2k\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") " pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:50:02.439897 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.439862    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:02.505267 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.505228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:50:02.505449 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.505366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:50:02.508134 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.508112    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e635ee7-8ae6-4a90-a66f-e33cfd434838-metrics-tls\") pod \"dns-default-wn4d2\" (UID: \"2e635ee7-8ae6-4a90-a66f-e33cfd434838\") " pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:50:02.508378 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.508351    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88805df0-8f06-432d-9316-c97536c8bc08-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-nj2nn\" (UID: \"88805df0-8f06-432d-9316-c97536c8bc08\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:50:02.561053 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.561007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:02.597046 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.597018    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mdgnt\""
Jul 16 21:50:02.604421 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.604394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:50:02.605847 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.605812    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sf8vj\""
Jul 16 21:50:02.613206 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.613186    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"
Jul 16 21:50:02.639420 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.639387    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ftbgv\""
Jul 16 21:50:02.646970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.646936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"
Jul 16 21:50:02.647272 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.647246    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:02.647839 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.647814    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pwsnw\""
Jul 16 21:50:02.656038 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.656018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"
Jul 16 21:50:02.665795 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.665778    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5gkfj\""
Jul 16 21:50:02.674185 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.674166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:02.676609 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.676592    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5kfkb\""
Jul 16 21:50:02.684978 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.684961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:50:02.950372 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:02.950286    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.028412 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.028379    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.141628 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.141584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.210707 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.210623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:50:03.211117 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.210730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:50:03.211327 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.211304    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:50:03.213403 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.213380    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38f4b3d2-9161-45a6-ad8c-cc0c509d65a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9dtrc\" (UID: \"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4\") " pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:50:03.227703 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.227674    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.297420 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.297387    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vv8qm\""
Jul 16 21:50:03.303828 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.303803    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9dtrc"
Jul 16 21:50:03.358813 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.358782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.548942 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.548672    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.634444 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.627369    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.688643 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.688548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wn4d2"]
Jul 16 21:50:03.692652 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.692521    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p"]
Jul 16 21:50:03.715436 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:50:03.715281    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e635ee7_8ae6_4a90_a66f_e33cfd434838.slice/crio-eebef19c62bee0fdd59e61a4117122c5de14a7268a7b83d490642057cac96e64 WatchSource:0}: Error finding container eebef19c62bee0fdd59e61a4117122c5de14a7268a7b83d490642057cac96e64: Status 404 returned error can't find the container with id eebef19c62bee0fdd59e61a4117122c5de14a7268a7b83d490642057cac96e64
Jul 16 21:50:03.739331 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.739298    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.842122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.842090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.950765 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.950735    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:03.951122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.951091    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn"]
Jul 16 21:50:03.952596 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.952548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758897b654-n8j2k"]
Jul 16 21:50:03.955068 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:50:03.955042    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88805df0_8f06_432d_9316_c97536c8bc08.slice/crio-ae0eabc205970d4e2846b8449494fbfd9c79e3036a4733c8ee770f92d865b41c WatchSource:0}: Error finding container ae0eabc205970d4e2846b8449494fbfd9c79e3036a4733c8ee770f92d865b41c: Status 404 returned error can't find the container with id ae0eabc205970d4e2846b8449494fbfd9c79e3036a4733c8ee770f92d865b41c
Jul 16 21:50:03.955546 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:50:03.955523    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84797633_98b5_4ce4_90ac_779d24813acf.slice/crio-769f44c1e15ee15a809d167561f138b9d00a54b801ed776a1898e6c67c9a9efd WatchSource:0}: Error finding container 769f44c1e15ee15a809d167561f138b9d00a54b801ed776a1898e6c67c9a9efd: Status 404 returned error can't find the container with id 769f44c1e15ee15a809d167561f138b9d00a54b801ed776a1898e6c67c9a9efd
Jul 16 21:50:03.955613 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.955578    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl"]
Jul 16 21:50:03.958509 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:50:03.958486    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6340ec6_1b89_4f62_a863_22358fc973ea.slice/crio-9bcba224e05ab8a5284605d8104f01a2b2fc334229df4b04c8c650481575c7c8 WatchSource:0}: Error finding container 9bcba224e05ab8a5284605d8104f01a2b2fc334229df4b04c8c650481575c7c8: Status 404 returned error can't find the container with id 9bcba224e05ab8a5284605d8104f01a2b2fc334229df4b04c8c650481575c7c8
Jul 16 21:50:03.962307 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.962197    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64db4fc64d-wlpxp"]
Jul 16 21:50:03.963835 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:03.963811    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9dtrc"]
Jul 16 21:50:03.964817 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:50:03.964801    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70792ad_fa1c_4f45_be6c_05892fd3fc19.slice/crio-f4be71f860158cc880a4e708e6cd54e7e5465e3004ae1f07ff8b435dd3a740ae WatchSource:0}: Error finding container f4be71f860158cc880a4e708e6cd54e7e5465e3004ae1f07ff8b435dd3a740ae: Status 404 returned error can't find the container with id f4be71f860158cc880a4e708e6cd54e7e5465e3004ae1f07ff8b435dd3a740ae
Jul 16 21:50:03.967900 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:50:03.967880    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f4b3d2_9161_45a6_ad8c_cc0c509d65a4.slice/crio-f0e0fa38ec6c8dbd908eee192a25bd16e6ee38ce1cec6983a175bb172448cbc0 WatchSource:0}: Error finding container f0e0fa38ec6c8dbd908eee192a25bd16e6ee38ce1cec6983a175bb172448cbc0: Status 404 returned error can't find the container with id f0e0fa38ec6c8dbd908eee192a25bd16e6ee38ce1cec6983a175bb172448cbc0
Jul 16 21:50:04.563973 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.563847    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:04.566199 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.566162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p" event={"ID":"b61aa2f1-c376-4a76-a60b-0ba45afdbd66","Type":"ContainerStarted","Data":"6e91aa37741be8a7e154d50bf0ec42a61f0de1b9a698e63a31d6201599370226"}
Jul 16 21:50:04.568445 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.568415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl" event={"ID":"f6340ec6-1b89-4f62-a863-22358fc973ea","Type":"ContainerStarted","Data":"9bcba224e05ab8a5284605d8104f01a2b2fc334229df4b04c8c650481575c7c8"}
Jul 16 21:50:04.570108 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.570082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wn4d2" event={"ID":"2e635ee7-8ae6-4a90-a66f-e33cfd434838","Type":"ContainerStarted","Data":"eebef19c62bee0fdd59e61a4117122c5de14a7268a7b83d490642057cac96e64"}
Jul 16 21:50:04.573122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.572233    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758897b654-n8j2k" event={"ID":"84797633-98b5-4ce4-90ac-779d24813acf","Type":"ContainerStarted","Data":"c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01"}
Jul 16 21:50:04.573122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.572277    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758897b654-n8j2k" event={"ID":"84797633-98b5-4ce4-90ac-779d24813acf","Type":"ContainerStarted","Data":"769f44c1e15ee15a809d167561f138b9d00a54b801ed776a1898e6c67c9a9efd"}
Jul 16 21:50:04.573122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.573049    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:50:04.577565 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.577501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn" event={"ID":"88805df0-8f06-432d-9316-c97536c8bc08","Type":"ContainerStarted","Data":"ae0eabc205970d4e2846b8449494fbfd9c79e3036a4733c8ee770f92d865b41c"}
Jul 16 21:50:04.586505 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.585084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64db4fc64d-wlpxp" event={"ID":"d70792ad-fa1c-4f45-be6c-05892fd3fc19","Type":"ContainerStarted","Data":"ce865e1d4eafb2ece7d1e725dfc01c72d1294cb2aabb7c221c5ab4d739f52dae"}
Jul 16 21:50:04.586505 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.585125    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64db4fc64d-wlpxp" event={"ID":"d70792ad-fa1c-4f45-be6c-05892fd3fc19","Type":"ContainerStarted","Data":"f4be71f860158cc880a4e708e6cd54e7e5465e3004ae1f07ff8b435dd3a740ae"}
Jul 16 21:50:04.590695 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.590653    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9dtrc" event={"ID":"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4","Type":"ContainerStarted","Data":"f0e0fa38ec6c8dbd908eee192a25bd16e6ee38ce1cec6983a175bb172448cbc0"}
Jul 16 21:50:04.592766 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.592741    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zkdf9" event={"ID":"5f9adec6-0471-441e-8c76-2dbdd18bfc18","Type":"ContainerStarted","Data":"d624af724d25650523896ed8025e58804a2c1c01175c04f8a3b90c75d9d4e113"}
Jul 16 21:50:04.636680 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.636622    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-758897b654-n8j2k" podStartSLOduration=69.636599944 podStartE2EDuration="1m9.636599944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:04.636204738 +0000 UTC m=+71.025478525" watchObservedRunningTime="2026-07-16 21:50:04.636599944 +0000 UTC m=+71.025873733"
Jul 16 21:50:04.674788 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.674756    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:04.678535 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.677216    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zkdf9" podStartSLOduration=65.007079976 podStartE2EDuration="1m9.677196826s" totalImagesPullingTime="4.67011685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:04.676776244 +0000 UTC m=+71.066050031" watchObservedRunningTime="2026-07-16 21:50:04.677196826 +0000 UTC m=+71.066470613"
Jul 16 21:50:04.678535 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.677978    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:04.800017 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.799951    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64db4fc64d-wlpxp" podStartSLOduration=68.799926917 podStartE2EDuration="1m8.799926917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:04.798179363 +0000 UTC m=+71.187453196" watchObservedRunningTime="2026-07-16 21:50:04.799926917 +0000 UTC m=+71.189200704"
Jul 16 21:50:04.997673 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:04.997634    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:05.232327 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:05.232262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:05.604734 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:05.604703    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:05.606628 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:05.606398    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64db4fc64d-wlpxp"
Jul 16 21:50:07.183518 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:07.183482    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:07.269462 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:07.269415    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:10.627737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.627650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn" event={"ID":"88805df0-8f06-432d-9316-c97536c8bc08","Type":"ContainerStarted","Data":"63932d493a5d089710b24ccf8a1cf2fae036ae288caa71998a6834d17d29ae48"}
Jul 16 21:50:10.629395 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.629370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p" event={"ID":"b61aa2f1-c376-4a76-a60b-0ba45afdbd66","Type":"ContainerStarted","Data":"8d3460a7f5f6fa2ac550a21ed59f8b8ad7504f388d6b01656a83303d71b413d6"}
Jul 16 21:50:10.631171 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.631143    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl" event={"ID":"f6340ec6-1b89-4f62-a863-22358fc973ea","Type":"ContainerStarted","Data":"5e88229a6bf617ad3c10ac02d2bd643bf4424e94436e50126c9aec4c65b6cf17"}
Jul 16 21:50:10.632790 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.632771    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wn4d2" event={"ID":"2e635ee7-8ae6-4a90-a66f-e33cfd434838","Type":"ContainerStarted","Data":"607f8976328e0e3420b636593ae037a8a213a99c17248e71cf4eb5c27a0d28c9"}
Jul 16 21:50:10.657853 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.657802    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-nj2nn" podStartSLOduration=37.334834099 podStartE2EDuration="43.657783568s" totalImagesPullingTime="6.322949469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:10.656963232 +0000 UTC m=+77.046237034" watchObservedRunningTime="2026-07-16 21:50:10.657783568 +0000 UTC m=+77.047057355"
Jul 16 21:50:10.686712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.686666    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-2f7rl" podStartSLOduration=68.367942119 podStartE2EDuration="1m14.68665033s" totalImagesPullingTime="6.318708211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:10.685968017 +0000 UTC m=+77.075241828" watchObservedRunningTime="2026-07-16 21:50:10.68665033 +0000 UTC m=+77.075924116"
Jul 16 21:50:10.766363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.766173    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:10.869394 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.869360    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:10.964801 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:10.964761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:11.046024 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:11.045973    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:11.640076 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:11.640034    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9dtrc" event={"ID":"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4","Type":"ContainerStarted","Data":"589a381e7e5977c53c6e8add6cb85bd5e6e40eb64895f9fce96713007876b2e9"}
Jul 16 21:50:11.642363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:11.642266    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p" event={"ID":"b61aa2f1-c376-4a76-a60b-0ba45afdbd66","Type":"ContainerStarted","Data":"ff5cde9f88b51f6e73d4a445e40020ce9d89afd4cd07d7ff8b5a085d5e9100c7"}
Jul 16 21:50:11.645274 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:11.645246    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wn4d2" event={"ID":"2e635ee7-8ae6-4a90-a66f-e33cfd434838","Type":"ContainerStarted","Data":"bbd907270c1861cb17175adf510875598c7fe6abea404c1c0f8d5d9226685b24"}
Jul 16 21:50:11.660750 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:11.660703    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8dg4p" podStartSLOduration=69.55079917 podStartE2EDuration="1m15.660688027s" totalImagesPullingTime="6.109888857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:11.659957556 +0000 UTC m=+78.049231374" watchObservedRunningTime="2026-07-16 21:50:11.660688027 +0000 UTC m=+78.049961939"
Jul 16 21:50:11.684443 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:11.684366    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wn4d2" podStartSLOduration=35.12651711 podStartE2EDuration="41.684346581s" totalImagesPullingTime="6.557829471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:11.683044684 +0000 UTC m=+78.072318469" watchObservedRunningTime="2026-07-16 21:50:11.684346581 +0000 UTC m=+78.073620369"
Jul 16 21:50:12.351279 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:12.351205    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:12.471842 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:12.471784    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:12.620796 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:12.620707    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:12.651987 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:12.651648    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:50:12.741640 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:12.741598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:14.194549 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:14.194499    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:14.287928 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:14.287874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:14.842961 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:14.842906    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:15.273629 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:15.273599    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:15.412252 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:15.412221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:15.662873 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:15.662790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9dtrc" event={"ID":"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4","Type":"ContainerStarted","Data":"271b2265b80ce6f3f7e63b0eda5df004b8cdc9b986bc32f4e931910c43397697"}
Jul 16 21:50:16.788408 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:16.788360    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:16.888935 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:16.888867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:19.680553 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:19.680518    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9dtrc" event={"ID":"38f4b3d2-9161-45a6-ad8c-cc0c509d65a4","Type":"ContainerStarted","Data":"a8bb9ccf1906617c6b6b4edbb4d214f3308f185ffa0b759a787fcd69b04b9d98"}
Jul 16 21:50:20.381189 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:20.381152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:21.090325 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:21.090285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:21.172075 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:21.172047    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:22.656501 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:22.656472    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wn4d2"
Jul 16 21:50:22.693964 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:22.693900    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9dtrc" podStartSLOduration=20.438171614 podStartE2EDuration="35.69388418s" totalImagesPullingTime="15.255712566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:49:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:19.704729979 +0000 UTC m=+86.094003766" watchObservedRunningTime="2026-07-16 21:50:22.69388418 +0000 UTC m=+89.083157966"
Jul 16 21:50:23.029134 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:23.029099    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:23.150567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:23.150528    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:23.353025 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:23.352934    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:25.196126 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:25.196090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:25.270713 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:25.270677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:26.389299 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:26.389269    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:26.455411 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:26.455382    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:26.612985 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:26.612954    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:50:26.941676 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:26.941639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:27.011518 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:27.011486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:30.526277 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:30.526249    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8ltd5"
Jul 16 21:50:31.050093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:31.050057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:31.146194 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:31.146162    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:31.848546 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:31.848511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:31.932678 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:31.932652    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:33.732696 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:33.732621    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:33.830701 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:33.830661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:33.930904 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:33.930871    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:35.167169 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:35.167136    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:35.245967 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:35.245931    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:35.725270 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:35.725238    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758897b654-n8j2k"]
Jul 16 21:50:35.822084 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:35.822053    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:35.921506 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:35.921476    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:36.510333 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.510301    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fjrps"]
Jul 16 21:50:36.513553 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.513526    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.513687 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.513579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:36.516338 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.516314    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 21:50:36.516463 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.516434    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jjtv5\""
Jul 16 21:50:36.516541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.516468    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 21:50:36.516541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.516484    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 21:50:36.516541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.516495    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 21:50:36.578790 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.578739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.578947 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.578812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-metrics-client-ca\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.578947 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.578844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-sys\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.578947 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.578868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-root\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.579073 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.578970    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-tls\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.579073 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.579010    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n89wr\" (UniqueName: \"kubernetes.io/projected/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-kube-api-access-n89wr\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.579073 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.579036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.579073 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.579070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-textfile\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.579201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.579090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-wtmp\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.599568 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.599539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:36.679571 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679545    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-textfile\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.679712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-wtmp\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.679712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.679712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-metrics-client-ca\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.679712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679660    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-sys\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.679712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-root\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.679712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-tls\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.679712 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n89wr\" (UniqueName: \"kubernetes.io/projected/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-kube-api-access-n89wr\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.680096 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.680096 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679763    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-root\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.680096 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679727    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-wtmp\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.680096 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-sys\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.680096 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:50:36.679816    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 16 21:50:36.680096 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:50:36.679894    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-tls podName:3c2df31e-e0d0-4429-a7e9-ec77871a0d9d nodeName:}" failed. No retries permitted until 2026-07-16 21:50:37.179873142 +0000 UTC m=+103.569146907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-tls") pod "node-exporter-fjrps" (UID: "3c2df31e-e0d0-4429-a7e9-ec77871a0d9d") : secret "node-exporter-tls" not found
Jul 16 21:50:36.680096 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.679904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-textfile\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.680431 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.680337    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.680431 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.680350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-metrics-client-ca\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.682189 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.682166    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:36.693416 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:36.693392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n89wr\" (UniqueName: \"kubernetes.io/projected/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-kube-api-access-n89wr\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:37.067023 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:37.066989    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:37.183988 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:37.183946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-tls\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:37.186394 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:37.186363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3c2df31e-e0d0-4429-a7e9-ec77871a0d9d-node-exporter-tls\") pod \"node-exporter-fjrps\" (UID: \"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d\") " pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:37.423285 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:37.423217    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fjrps"
Jul 16 21:50:37.477929 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:50:37.477883    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c2df31e_e0d0_4429_a7e9_ec77871a0d9d.slice/crio-a89bd11f7d6bdf5f9ffa00c6828a271ef18165c27efcbe1bd2b1c22a0b3c9e60 WatchSource:0}: Error finding container a89bd11f7d6bdf5f9ffa00c6828a271ef18165c27efcbe1bd2b1c22a0b3c9e60: Status 404 returned error can't find the container with id a89bd11f7d6bdf5f9ffa00c6828a271ef18165c27efcbe1bd2b1c22a0b3c9e60
Jul 16 21:50:37.543827 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:37.543796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:37.742995 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:37.742955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:37.745672 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:37.745648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjrps" event={"ID":"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d","Type":"ContainerStarted","Data":"a89bd11f7d6bdf5f9ffa00c6828a271ef18165c27efcbe1bd2b1c22a0b3c9e60"}
Jul 16 21:50:38.480086 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:38.480047    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:39.422073 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:39.421981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:39.515626 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:39.515575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:39.759085 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:39.759057    2564 generic.go:356] "Generic (PLEG): container finished" podID="3c2df31e-e0d0-4429-a7e9-ec77871a0d9d" containerID="beb5c9dea049962afb8baefc9987d387d01d2e0a1b7dd3aafd5053edd8ba7901" exitCode=0
Jul 16 21:50:39.759270 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:39.759150    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjrps" event={"ID":"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d","Type":"ContainerDied","Data":"beb5c9dea049962afb8baefc9987d387d01d2e0a1b7dd3aafd5053edd8ba7901"}
Jul 16 21:50:40.096670 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:40.096581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:40.219515 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:40.219484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:40.765446 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:40.765411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjrps" event={"ID":"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d","Type":"ContainerStarted","Data":"64fae0f74af53075f85b802009fc01b6f20a9ca7ed13dfd3ceb6e1fdaee8bd5a"}
Jul 16 21:50:40.765929 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:40.765464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjrps" event={"ID":"3c2df31e-e0d0-4429-a7e9-ec77871a0d9d","Type":"ContainerStarted","Data":"da33663ca31203f0ddad7b19e875ff8b3f8655244872ab945add3e7c21185a78"}
Jul 16 21:50:40.789154 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:40.789096    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fjrps" podStartSLOduration=3.079684918 podStartE2EDuration="4.789076092s" totalImagesPullingTime="1.611081345s" totalInitContainerRuntime="98.309829ms" isStatefulPod=true podCreationTimestamp="2026-07-16 21:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:50:40.787479154 +0000 UTC m=+107.176752939" watchObservedRunningTime="2026-07-16 21:50:40.789076092 +0000 UTC m=+107.178349879"
Jul 16 21:50:41.190145 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:41.190098    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:41.278275 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:41.278238    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:41.386063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:41.386028    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:41.484118 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:41.484035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:42.837600 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:42.837554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:42.996478 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:42.996440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:52.278184 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:52.278148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:52.395994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:52.395959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:58.424763 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:58.424728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:58.529105 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:58.529067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:50:58.629785 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:50:58.629747    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:00.745704 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:00.745660    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-758897b654-n8j2k" podUID="84797633-98b5-4ce4-90ac-779d24813acf" containerName="registry" containerID="cri-o://c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01" gracePeriod=30
Jul 16 21:51:00.837366 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:00.837333    2564 generic.go:356] "Generic (PLEG): container finished" podID="748c76b0-d147-4d51-9377-9409e9faefa9" containerID="3d6519d3a10bef6f9c81c6f61e908dec6ab988e10cccd3c891f4574309f31c61" exitCode=0
Jul 16 21:51:00.837496 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:00.837401    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4" event={"ID":"748c76b0-d147-4d51-9377-9409e9faefa9","Type":"ContainerDied","Data":"3d6519d3a10bef6f9c81c6f61e908dec6ab988e10cccd3c891f4574309f31c61"}
Jul 16 21:51:00.837737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:00.837722    2564 scope.go:117] "RemoveContainer" containerID="3d6519d3a10bef6f9c81c6f61e908dec6ab988e10cccd3c891f4574309f31c61"
Jul 16 21:51:01.001359 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.001307    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:51:01.088961 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.088889    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-image-registry-private-configuration\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089131 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.088992    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-bound-sa-token\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089131 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089046    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-trusted-ca\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089131 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089087    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-installation-pull-secrets\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089131 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089128    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jdjp\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-kube-api-access-6jdjp\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089339 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089158    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84797633-98b5-4ce4-90ac-779d24813acf-ca-trust-extracted\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089339 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089183    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089339 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089216    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-registry-certificates\") pod \"84797633-98b5-4ce4-90ac-779d24813acf\" (UID: \"84797633-98b5-4ce4-90ac-779d24813acf\") "
Jul 16 21:51:01.089760 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:51:01.089856 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.089770    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:51:01.092030 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.092001    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:51:01.092137 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.092044    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:51:01.092137 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.092103    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-kube-api-access-6jdjp" (OuterVolumeSpecName: "kube-api-access-6jdjp") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "kube-api-access-6jdjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:51:01.092237 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.092217    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:51:01.092333 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.092311    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:51:01.098023 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.097998    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84797633-98b5-4ce4-90ac-779d24813acf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "84797633-98b5-4ce4-90ac-779d24813acf" (UID: "84797633-98b5-4ce4-90ac-779d24813acf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:51:01.190225 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190200    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-image-registry-private-configuration\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.190225 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190222    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-bound-sa-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.190225 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190231    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-trusted-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.190434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190240    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84797633-98b5-4ce4-90ac-779d24813acf-installation-pull-secrets\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.190434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190248    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jdjp\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-kube-api-access-6jdjp\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.190434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190268    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84797633-98b5-4ce4-90ac-779d24813acf-ca-trust-extracted\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.190434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190277    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84797633-98b5-4ce4-90ac-779d24813acf-registry-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.190434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.190285    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84797633-98b5-4ce4-90ac-779d24813acf-registry-certificates\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:51:01.842110 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.842072    2564 generic.go:356] "Generic (PLEG): container finished" podID="84797633-98b5-4ce4-90ac-779d24813acf" containerID="c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01" exitCode=0
Jul 16 21:51:01.842521 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.842121    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758897b654-n8j2k"
Jul 16 21:51:01.842521 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.842164    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758897b654-n8j2k" event={"ID":"84797633-98b5-4ce4-90ac-779d24813acf","Type":"ContainerDied","Data":"c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01"}
Jul 16 21:51:01.842521 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.842214    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758897b654-n8j2k" event={"ID":"84797633-98b5-4ce4-90ac-779d24813acf","Type":"ContainerDied","Data":"769f44c1e15ee15a809d167561f138b9d00a54b801ed776a1898e6c67c9a9efd"}
Jul 16 21:51:01.842521 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.842237    2564 scope.go:117] "RemoveContainer" containerID="c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01"
Jul 16 21:51:01.844220 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.844186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qxdl4" event={"ID":"748c76b0-d147-4d51-9377-9409e9faefa9","Type":"ContainerStarted","Data":"e86103a389ca584aab371d5d0a0ad0f14b3381103a4b456240236f019cc1b45f"}
Jul 16 21:51:01.850940 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.850900    2564 scope.go:117] "RemoveContainer" containerID="c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01"
Jul 16 21:51:01.851230 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:51:01.851210    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01\": container with ID starting with c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01 not found: ID does not exist" containerID="c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01"
Jul 16 21:51:01.851293 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.851237    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01"} err="failed to get container status \"c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01\": rpc error: code = NotFound desc = could not find container \"c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01\": container with ID starting with c3b1df450aafebba7032cb1cec96cf5f8ecf745fe83b544f4edb4c6211abce01 not found: ID does not exist"
Jul 16 21:51:01.881086 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.881057    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758897b654-n8j2k"]
Jul 16 21:51:01.889493 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:01.889466    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-758897b654-n8j2k"]
Jul 16 21:51:02.124029 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:02.123960    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84797633-98b5-4ce4-90ac-779d24813acf" path="/var/lib/kubelet/pods/84797633-98b5-4ce4-90ac-779d24813acf/volumes"
Jul 16 21:51:02.805264 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:02.805217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:51:02.807664 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:02.807642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d31ef25-6751-41d8-9583-11a3455909e6-metrics-certs\") pod \"network-metrics-daemon-hzwzf\" (UID: \"9d31ef25-6751-41d8-9583-11a3455909e6\") " pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:51:03.050896 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:03.050868    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xnvp2\""
Jul 16 21:51:03.058556 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:03.058502    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hzwzf"
Jul 16 21:51:03.193726 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:03.193702    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hzwzf"]
Jul 16 21:51:03.195899 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:51:03.195870    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d31ef25_6751_41d8_9583_11a3455909e6.slice/crio-7eef81daf5e8749bafa9c50859a689b07bf9a1e1cbbe09d961498a8289bfe7fe WatchSource:0}: Error finding container 7eef81daf5e8749bafa9c50859a689b07bf9a1e1cbbe09d961498a8289bfe7fe: Status 404 returned error can't find the container with id 7eef81daf5e8749bafa9c50859a689b07bf9a1e1cbbe09d961498a8289bfe7fe
Jul 16 21:51:03.856778 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:03.856748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hzwzf" event={"ID":"9d31ef25-6751-41d8-9583-11a3455909e6","Type":"ContainerStarted","Data":"7eef81daf5e8749bafa9c50859a689b07bf9a1e1cbbe09d961498a8289bfe7fe"}
Jul 16 21:51:04.862806 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:04.862771    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hzwzf" event={"ID":"9d31ef25-6751-41d8-9583-11a3455909e6","Type":"ContainerStarted","Data":"9d943ea61fde7f9f365988970eeea3f82400ae5a1d04ee9008bfb8ea9a3a14d6"}
Jul 16 21:51:04.863244 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:04.862854    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hzwzf" event={"ID":"9d31ef25-6751-41d8-9583-11a3455909e6","Type":"ContainerStarted","Data":"8a54cb8e010c6a427ead3fe443ff7e29f2aa63b8cc769377808fd7d53b373571"}
Jul 16 21:51:04.884043 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:04.883997    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hzwzf" podStartSLOduration=129.65537358 podStartE2EDuration="2m10.883982319s" totalImagesPullingTime="1.228608739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:04.88159857 +0000 UTC m=+131.270872380" watchObservedRunningTime="2026-07-16 21:51:04.883982319 +0000 UTC m=+131.273256112"
Jul 16 21:51:05.006452 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:05.006417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:05.620621 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:05.620587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:06.872796 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:06.872763    2564 generic.go:356] "Generic (PLEG): container finished" podID="1173fb03-41c1-4aad-902f-7b7f6d9bd371" containerID="ca83155c565ebfdecd6e214d74d0e00f5c87ccd12caf70396214c11a9ac7eaf3" exitCode=0
Jul 16 21:51:06.873209 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:06.872812    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-trdvm" event={"ID":"1173fb03-41c1-4aad-902f-7b7f6d9bd371","Type":"ContainerDied","Data":"ca83155c565ebfdecd6e214d74d0e00f5c87ccd12caf70396214c11a9ac7eaf3"}
Jul 16 21:51:06.873209 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:06.873134    2564 scope.go:117] "RemoveContainer" containerID="ca83155c565ebfdecd6e214d74d0e00f5c87ccd12caf70396214c11a9ac7eaf3"
Jul 16 21:51:07.431700 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:07.431671    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64db4fc64d-wlpxp_d70792ad-fa1c-4f45-be6c-05892fd3fc19/router/0.log"
Jul 16 21:51:07.439027 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:07.439004    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8pnfs_3b323658-e5c3-4845-9038-a8695bfed36f/serve-healthcheck-canary/0.log"
Jul 16 21:51:07.878984 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:07.878950    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-trdvm" event={"ID":"1173fb03-41c1-4aad-902f-7b7f6d9bd371","Type":"ContainerStarted","Data":"2a9f911d7263571a5e23022a48e0075666d4fe9a5f4671a112c768ef068218b6"}
Jul 16 21:51:10.892373 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:10.892331    2564 generic.go:356] "Generic (PLEG): container finished" podID="52af9bbb-a233-445a-98e6-22f0cc478c60" containerID="4818d9b124c42c76809eae4c8cf08bf9baf7a9946a69c244e2383811efb6eb3f" exitCode=0
Jul 16 21:51:10.892794 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:10.892394    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b" event={"ID":"52af9bbb-a233-445a-98e6-22f0cc478c60","Type":"ContainerDied","Data":"4818d9b124c42c76809eae4c8cf08bf9baf7a9946a69c244e2383811efb6eb3f"}
Jul 16 21:51:10.892794 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:10.892708    2564 scope.go:117] "RemoveContainer" containerID="4818d9b124c42c76809eae4c8cf08bf9baf7a9946a69c244e2383811efb6eb3f"
Jul 16 21:51:11.898065 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:11.898031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jr92b" event={"ID":"52af9bbb-a233-445a-98e6-22f0cc478c60","Type":"ContainerStarted","Data":"7d82fb0eb8bbbfbef2b3e8e570bc6ad1ea7238c66644fb36cbff8c55f3e6bfc1"}
Jul 16 21:51:13.075830 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:13.075786    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" podUID="3af01737-a87a-493c-8b1e-955d4d51d03a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 16 21:51:14.575158 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:14.575116    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:14.745016 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:14.744977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:14.840753 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:14.840668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:23.075178 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:23.075141    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" podUID="3af01737-a87a-493c-8b1e-955d4d51d03a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 16 21:51:33.075122 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:33.075081    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" podUID="3af01737-a87a-493c-8b1e-955d4d51d03a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 16 21:51:33.075507 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:33.075151    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk"
Jul 16 21:51:33.075603 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:33.075585    2564 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"b33d8388e296ed2d6c3d4690a7539b209f5b1dd21fa4ad666fdd8cfb9df1d88b"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 16 21:51:33.075639 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:33.075623    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" podUID="3af01737-a87a-493c-8b1e-955d4d51d03a" containerName="service-proxy" containerID="cri-o://b33d8388e296ed2d6c3d4690a7539b209f5b1dd21fa4ad666fdd8cfb9df1d88b" gracePeriod=30
Jul 16 21:51:33.981389 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:33.981360    2564 generic.go:356] "Generic (PLEG): container finished" podID="3af01737-a87a-493c-8b1e-955d4d51d03a" containerID="b33d8388e296ed2d6c3d4690a7539b209f5b1dd21fa4ad666fdd8cfb9df1d88b" exitCode=2
Jul 16 21:51:33.981580 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:33.981440    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" event={"ID":"3af01737-a87a-493c-8b1e-955d4d51d03a","Type":"ContainerDied","Data":"b33d8388e296ed2d6c3d4690a7539b209f5b1dd21fa4ad666fdd8cfb9df1d88b"}
Jul 16 21:51:33.981580 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:33.981493    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6766cb4db4-v89lk" event={"ID":"3af01737-a87a-493c-8b1e-955d4d51d03a","Type":"ContainerStarted","Data":"825dc23deab4504283b82887b0337fb8c3f951fba44b88a32611417e19465bda"}
Jul 16 21:51:46.184516 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:46.184434    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:52.679063 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:52.679007    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:51:52.794488 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:51:52.794459    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:02.168074 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:02.168011    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:02.233130 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:02.233097    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:04.612348 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:04.612308    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:04.676132 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:04.676101    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:07.173809 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:07.173771    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:07.275025 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:07.274998    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:12.703075 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:12.703035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:12.804824 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:12.804790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:12.877259 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:12.877220    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:32.336326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:32.336263    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:32.433316 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:32.433279    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:37.534979 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:37.534936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:37.591226 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:37.591192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:49.022354 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:49.022316    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:54.292877 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:54.292838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:54.626900 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:54.626768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:57.956550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:57.956507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:58.284282 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:58.284169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:58.385639 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:58.385604    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:59.277987 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:59.277949    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:59.524869 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:59.524829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:52:59.732958 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:52:59.732895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:01.762032 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:01.761967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:02.226690 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:02.226657    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:04.869036 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:04.869001    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:04.935721 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:04.935672    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:08.275288 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:08.275251    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:08.377246 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:08.377190    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:10.331887 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:10.331843    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:13.142189 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:13.142156    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:13.220645 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:13.220608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:14.628536 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:14.628429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:14.729335 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:14.729303    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:16.671059 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:16.671015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:32.949834 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:32.949792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:33.109579 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:33.109543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:33.571835 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:33.571803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:33.955465 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:33.955424    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:34.114731 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.114686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:34.171990 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.171955    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"]
Jul 16 21:53:34.172230 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.172218    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84797633-98b5-4ce4-90ac-779d24813acf" containerName="registry"
Jul 16 21:53:34.172283 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.172232    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="84797633-98b5-4ce4-90ac-779d24813acf" containerName="registry"
Jul 16 21:53:34.172326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.172292    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="84797633-98b5-4ce4-90ac-779d24813acf" containerName="registry"
Jul 16 21:53:34.176991 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.176952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.180389 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.180369    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 21:53:34.181109 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.181091    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 21:53:34.181388 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.181368    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 21:53:34.181487 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.181414    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 21:53:34.181559 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.181499    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hk7b8\""
Jul 16 21:53:34.199659 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.199631    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"]
Jul 16 21:53:34.266646 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.266565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58d9f09a-6837-4e79-b0c0-78f830eeb47d-webhook-cert\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.266646 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.266600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/58d9f09a-6837-4e79-b0c0-78f830eeb47d-manifests\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.266646 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.266627    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/58d9f09a-6837-4e79-b0c0-78f830eeb47d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.266840 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.266682    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58d9f09a-6837-4e79-b0c0-78f830eeb47d-tmp\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.266840 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.266720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wn6\" (UniqueName: \"kubernetes.io/projected/58d9f09a-6837-4e79-b0c0-78f830eeb47d-kube-api-access-s2wn6\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.354025 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.353986    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:34.367017 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.366997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wn6\" (UniqueName: \"kubernetes.io/projected/58d9f09a-6837-4e79-b0c0-78f830eeb47d-kube-api-access-s2wn6\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.367104 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.367032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58d9f09a-6837-4e79-b0c0-78f830eeb47d-webhook-cert\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.367104 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.367066    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/58d9f09a-6837-4e79-b0c0-78f830eeb47d-manifests\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.367176 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.367105    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/58d9f09a-6837-4e79-b0c0-78f830eeb47d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.367176 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.367137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58d9f09a-6837-4e79-b0c0-78f830eeb47d-tmp\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.367560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.367539    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/58d9f09a-6837-4e79-b0c0-78f830eeb47d-manifests\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.369465 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.369443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58d9f09a-6837-4e79-b0c0-78f830eeb47d-tmp\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.369589 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.369571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58d9f09a-6837-4e79-b0c0-78f830eeb47d-webhook-cert\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.369717 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.369699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/58d9f09a-6837-4e79-b0c0-78f830eeb47d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.376367 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.376345    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wn6\" (UniqueName: \"kubernetes.io/projected/58d9f09a-6837-4e79-b0c0-78f830eeb47d-kube-api-access-s2wn6\") pod \"opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k\" (UID: \"58d9f09a-6837-4e79-b0c0-78f830eeb47d\") " pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.473674 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.473640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:34.487509 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.487485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:34.639092 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.639061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"]
Jul 16 21:53:34.642753 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:53:34.642724    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d9f09a_6837_4e79_b0c0_78f830eeb47d.slice/crio-d266bb9c0fef03abcc96134dff143299c6b84fbf8cf2d0c69c7e340781291f67 WatchSource:0}: Error finding container d266bb9c0fef03abcc96134dff143299c6b84fbf8cf2d0c69c7e340781291f67: Status 404 returned error can't find the container with id d266bb9c0fef03abcc96134dff143299c6b84fbf8cf2d0c69c7e340781291f67
Jul 16 21:53:34.654076 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.654046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:34.744463 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.744433    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:34.852984 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.852874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:34.951405 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:34.951377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:35.056774 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:35.056736    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:35.207281 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:35.207248    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:35.427762 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:35.427731    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k" event={"ID":"58d9f09a-6837-4e79-b0c0-78f830eeb47d","Type":"ContainerStarted","Data":"d266bb9c0fef03abcc96134dff143299c6b84fbf8cf2d0c69c7e340781291f67"}
Jul 16 21:53:38.443775 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:38.443740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k" event={"ID":"58d9f09a-6837-4e79-b0c0-78f830eeb47d","Type":"ContainerStarted","Data":"1c6f7f8285f1bfd574731235f688f0fd6814c2daba19594366a91fc73af6425d"}
Jul 16 21:53:39.457268 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:39.457235    2564 generic.go:356] "Generic (PLEG): container finished" podID="58d9f09a-6837-4e79-b0c0-78f830eeb47d" containerID="1c6f7f8285f1bfd574731235f688f0fd6814c2daba19594366a91fc73af6425d" exitCode=0
Jul 16 21:53:39.457636 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:39.457287    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k" event={"ID":"58d9f09a-6837-4e79-b0c0-78f830eeb47d","Type":"ContainerDied","Data":"1c6f7f8285f1bfd574731235f688f0fd6814c2daba19594366a91fc73af6425d"}
Jul 16 21:53:40.464257 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:40.464208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k" event={"ID":"58d9f09a-6837-4e79-b0c0-78f830eeb47d","Type":"ContainerStarted","Data":"f98e36ff9f0a5d2bba62951338e2b24f71db1119404114e9ccec1b7c100618af"}
Jul 16 21:53:40.464690 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:40.464295    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:40.501294 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:40.501234    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k" podStartSLOduration=1.888378143 podStartE2EDuration="6.50121845s" totalImagesPullingTime="3.580453252s" totalInitContainerRuntime="1.032387055s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:53:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:53:40.50017358 +0000 UTC m=+286.889447367" watchObservedRunningTime="2026-07-16 21:53:40.50121845 +0000 UTC m=+286.890492235"
Jul 16 21:53:45.547154 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:45.547104    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:47.023890 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:47.023839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:51.470808 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:51.470777    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k"
Jul 16 21:53:53.485688 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:53.485650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:53.699323 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:53.699284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:54.003027 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.002997    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 21:53:54.011180 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.011146    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 21:53:54.016564 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.016539    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:53:54.019178 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.019159    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:53:54.020270 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.020253    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 21:53:54.556906 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.556869    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:54.699416 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.699381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:54.846250 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:54.846171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:55.010608 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:55.010576    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:55.581785 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:55.581755    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:56.025448 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:56.025410    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:57.426168 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:57.426130    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:57.525632 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:57.525598    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:57.800000 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:57.799961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:57.989094 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:57.989058    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:59.826329 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.826292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:53:59.935996 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.935969    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"]
Jul 16 21:53:59.939257 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.939238    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:53:59.943423 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.943402    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 21:53:59.943526 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.943406    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l5xts\""
Jul 16 21:53:59.943526 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.943452    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 21:53:59.955224 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.955203    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"]
Jul 16 21:53:59.970187 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:53:59.970150    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:00.065513 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.065478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28f7e0b8-1a4d-4187-bc78-058925285ebe-tls-certs\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.065665 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.065518    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxc66\" (UniqueName: \"kubernetes.io/projected/28f7e0b8-1a4d-4187-bc78-058925285ebe-kube-api-access-wxc66\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.065665 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.065613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28f7e0b8-1a4d-4187-bc78-058925285ebe-tmp\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.166848 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.166774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28f7e0b8-1a4d-4187-bc78-058925285ebe-tls-certs\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.166848 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.166812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxc66\" (UniqueName: \"kubernetes.io/projected/28f7e0b8-1a4d-4187-bc78-058925285ebe-kube-api-access-wxc66\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.167053 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.166873    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28f7e0b8-1a4d-4187-bc78-058925285ebe-tmp\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.169195 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.169173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28f7e0b8-1a4d-4187-bc78-058925285ebe-tmp\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.169521 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.169499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28f7e0b8-1a4d-4187-bc78-058925285ebe-tls-certs\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.176964 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.176941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxc66\" (UniqueName: \"kubernetes.io/projected/28f7e0b8-1a4d-4187-bc78-058925285ebe-kube-api-access-wxc66\") pod \"kube-auth-proxy-695b65bf64-rfgxf\" (UID: \"28f7e0b8-1a4d-4187-bc78-058925285ebe\") " pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.248807 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.248761    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"
Jul 16 21:54:00.407171 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.407130    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf"]
Jul 16 21:54:00.412219 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:54:00.412195    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f7e0b8_1a4d_4187_bc78_058925285ebe.slice/crio-3894d4cbd43080327b82d3e28873578c037d78cd2489fa0bb93ea62c8a3456d2 WatchSource:0}: Error finding container 3894d4cbd43080327b82d3e28873578c037d78cd2489fa0bb93ea62c8a3456d2: Status 404 returned error can't find the container with id 3894d4cbd43080327b82d3e28873578c037d78cd2489fa0bb93ea62c8a3456d2
Jul 16 21:54:00.414057 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.414038    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:54:00.546636 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:00.546606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf" event={"ID":"28f7e0b8-1a4d-4187-bc78-058925285ebe","Type":"ContainerStarted","Data":"3894d4cbd43080327b82d3e28873578c037d78cd2489fa0bb93ea62c8a3456d2"}
Jul 16 21:54:02.119457 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.119426    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"]
Jul 16 21:54:02.123138 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.123112    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.129374 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.129352    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 21:54:02.129517 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.129457    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 21:54:02.129613 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.129354    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 21:54:02.129742 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.129724    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 21:54:02.133768 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.133751    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zqvqp\""
Jul 16 21:54:02.157460 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.157416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:02.183399 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.183362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/535094a3-8021-4020-b553-b88951266cdb-local-certs\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.183567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.183427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-istio-token\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.183567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.183467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-cacerts\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.183567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.183494    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.183567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.183527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.183567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.183563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/535094a3-8021-4020-b553-b88951266cdb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.183807 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.183597    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkxqr\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-kube-api-access-dkxqr\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.282532 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.282495    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"]
Jul 16 21:54:02.284639 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.284610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/535094a3-8021-4020-b553-b88951266cdb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.284788 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.284663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkxqr\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-kube-api-access-dkxqr\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.284788 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.284758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/535094a3-8021-4020-b553-b88951266cdb-local-certs\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.284891 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.284803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-istio-token\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.284891 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.284842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-cacerts\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.284891 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.284865    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.285065 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.284889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.285445 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.285393    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/535094a3-8021-4020-b553-b88951266cdb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.288041 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.287941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/535094a3-8021-4020-b553-b88951266cdb-local-certs\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.288188 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.288153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-cacerts\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.288274 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.288161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.288529 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.288507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.309934 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.309894    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-istio-token\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.334892 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.334862    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkxqr\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-kube-api-access-dkxqr\") pod \"istiod-openshift-gateway-7574459656-khbnv\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.378393 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.378306    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:02.435372 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.435350    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:02.563246 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.563204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf" event={"ID":"28f7e0b8-1a4d-4187-bc78-058925285ebe","Type":"ContainerStarted","Data":"c810cb3d9573f409a17236a2e0a945f06678c21dc8426a1f399f855d6fee5f4e"}
Jul 16 21:54:02.587709 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.587683    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"]
Jul 16 21:54:02.590861 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.590829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:02.591069 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:54:02.591048    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535094a3_8021_4020_b553_b88951266cdb.slice/crio-fab42dfb6ff358b4efcfeac7a74a6a8a78e3d432a02ea1017d67f841c95ee5f5 WatchSource:0}: Error finding container fab42dfb6ff358b4efcfeac7a74a6a8a78e3d432a02ea1017d67f841c95ee5f5: Status 404 returned error can't find the container with id fab42dfb6ff358b4efcfeac7a74a6a8a78e3d432a02ea1017d67f841c95ee5f5
Jul 16 21:54:02.610195 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:02.610148    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-695b65bf64-rfgxf" podStartSLOduration=1.644527235 podStartE2EDuration="3.610132268s" totalImagesPullingTime="1.965605033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:02.608758708 +0000 UTC m=+308.998032493" watchObservedRunningTime="2026-07-16 21:54:02.610132268 +0000 UTC m=+308.999406053"
Jul 16 21:54:03.570137 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:03.570070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv" event={"ID":"535094a3-8021-4020-b553-b88951266cdb","Type":"ContainerStarted","Data":"fab42dfb6ff358b4efcfeac7a74a6a8a78e3d432a02ea1017d67f841c95ee5f5"}
Jul 16 21:54:05.164480 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:05.164444    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 16 21:54:05.164772 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:05.164509    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 16 21:54:05.486095 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:05.486063    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:05.580461 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:05.580429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv" event={"ID":"535094a3-8021-4020-b553-b88951266cdb","Type":"ContainerStarted","Data":"1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864"}
Jul 16 21:54:05.580628 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:05.580555    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:05.609421 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:05.609370    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv" podStartSLOduration=1.038070837 podStartE2EDuration="3.609352311s" totalImagesPullingTime="2.571281474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:05.607935676 +0000 UTC m=+311.997209476" watchObservedRunningTime="2026-07-16 21:54:05.609352311 +0000 UTC m=+311.998626098"
Jul 16 21:54:05.813846 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:05.813679    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:06.031629 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:06.031587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:06.586754 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:06.586721    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-khbnv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:54:06.587156 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:06.586778    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv" podUID="535094a3-8021-4020-b553-b88951266cdb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:54:07.409253 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.409222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:07.660544 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.660459    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"]
Jul 16 21:54:07.663857 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.663835    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:07.667991 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.667972    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-26dwh\""
Jul 16 21:54:07.668093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.668005    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 16 21:54:07.691157 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.691137    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"]
Jul 16 21:54:07.694204 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.694187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:07.700311 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.700283    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 16 21:54:07.703111 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.703091    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"]
Jul 16 21:54:07.704734 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.704704    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nvk6f\""
Jul 16 21:54:07.730970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.730944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd872a79-78f1-4c79-b884-6b2723da3867-cert\") pod \"kserve-controller-manager-d6f4b65b8-tlhk5\" (UID: \"cd872a79-78f1-4c79-b884-6b2723da3867\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:07.731072 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.730979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2h85\" (UniqueName: \"kubernetes.io/projected/cd872a79-78f1-4c79-b884-6b2723da3867-kube-api-access-m2h85\") pod \"kserve-controller-manager-d6f4b65b8-tlhk5\" (UID: \"cd872a79-78f1-4c79-b884-6b2723da3867\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:07.744759 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.744725    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"]
Jul 16 21:54:07.797187 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.797152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:07.832026 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.831999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd872a79-78f1-4c79-b884-6b2723da3867-cert\") pod \"kserve-controller-manager-d6f4b65b8-tlhk5\" (UID: \"cd872a79-78f1-4c79-b884-6b2723da3867\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:07.832140 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.832033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8lhg\" (UniqueName: \"kubernetes.io/projected/0653006a-640b-4264-be65-9c922d3d7f3a-kube-api-access-q8lhg\") pod \"llmisvc-controller-manager-777c6bd7d5-bvzc8\" (UID: \"0653006a-640b-4264-be65-9c922d3d7f3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:07.832140 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.832059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2h85\" (UniqueName: \"kubernetes.io/projected/cd872a79-78f1-4c79-b884-6b2723da3867-kube-api-access-m2h85\") pod \"kserve-controller-manager-d6f4b65b8-tlhk5\" (UID: \"cd872a79-78f1-4c79-b884-6b2723da3867\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:07.832140 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.832094    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0653006a-640b-4264-be65-9c922d3d7f3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bvzc8\" (UID: \"0653006a-640b-4264-be65-9c922d3d7f3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:07.834650 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.834630    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd872a79-78f1-4c79-b884-6b2723da3867-cert\") pod \"kserve-controller-manager-d6f4b65b8-tlhk5\" (UID: \"cd872a79-78f1-4c79-b884-6b2723da3867\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:07.859552 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.859523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2h85\" (UniqueName: \"kubernetes.io/projected/cd872a79-78f1-4c79-b884-6b2723da3867-kube-api-access-m2h85\") pod \"kserve-controller-manager-d6f4b65b8-tlhk5\" (UID: \"cd872a79-78f1-4c79-b884-6b2723da3867\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:07.932572 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.932497    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8lhg\" (UniqueName: \"kubernetes.io/projected/0653006a-640b-4264-be65-9c922d3d7f3a-kube-api-access-q8lhg\") pod \"llmisvc-controller-manager-777c6bd7d5-bvzc8\" (UID: \"0653006a-640b-4264-be65-9c922d3d7f3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:07.932572 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.932562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0653006a-640b-4264-be65-9c922d3d7f3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bvzc8\" (UID: \"0653006a-640b-4264-be65-9c922d3d7f3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:07.935086 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.935063    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0653006a-640b-4264-be65-9c922d3d7f3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bvzc8\" (UID: \"0653006a-640b-4264-be65-9c922d3d7f3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:07.943962 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.943940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8lhg\" (UniqueName: \"kubernetes.io/projected/0653006a-640b-4264-be65-9c922d3d7f3a-kube-api-access-q8lhg\") pod \"llmisvc-controller-manager-777c6bd7d5-bvzc8\" (UID: \"0653006a-640b-4264-be65-9c922d3d7f3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:07.945723 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.945692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:07.973790 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:07.973769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:08.010713 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:08.010659    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:08.135542 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:08.135440    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"]
Jul 16 21:54:08.137796 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:54:08.137764    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd872a79_78f1_4c79_b884_6b2723da3867.slice/crio-66d319fc1689b69e43f88c46b7b514091d0ce8086d36e518db14174f9d1e9898 WatchSource:0}: Error finding container 66d319fc1689b69e43f88c46b7b514091d0ce8086d36e518db14174f9d1e9898: Status 404 returned error can't find the container with id 66d319fc1689b69e43f88c46b7b514091d0ce8086d36e518db14174f9d1e9898
Jul 16 21:54:08.181058 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:08.181025    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"]
Jul 16 21:54:08.184615 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:54:08.184554    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0653006a_640b_4264_be65_9c922d3d7f3a.slice/crio-fc9ca0d54fc93df835f9b2baeccadc7b29fb102e4587e4513b0ec00a42a56bdd WatchSource:0}: Error finding container fc9ca0d54fc93df835f9b2baeccadc7b29fb102e4587e4513b0ec00a42a56bdd: Status 404 returned error can't find the container with id fc9ca0d54fc93df835f9b2baeccadc7b29fb102e4587e4513b0ec00a42a56bdd
Jul 16 21:54:08.594381 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:08.594334    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8" event={"ID":"0653006a-640b-4264-be65-9c922d3d7f3a","Type":"ContainerStarted","Data":"fc9ca0d54fc93df835f9b2baeccadc7b29fb102e4587e4513b0ec00a42a56bdd"}
Jul 16 21:54:08.595452 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:08.595425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5" event={"ID":"cd872a79-78f1-4c79-b884-6b2723da3867","Type":"ContainerStarted","Data":"66d319fc1689b69e43f88c46b7b514091d0ce8086d36e518db14174f9d1e9898"}
Jul 16 21:54:08.849339 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:08.849254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:09.049184 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:09.049153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:09.143390 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:09.143179    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:09.586047 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:09.586016    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:09.735422 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:09.735379    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:09.835604 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:09.835561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:10.295858 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:10.295805    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"]
Jul 16 21:54:10.296351 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:10.296058    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv" podUID="535094a3-8021-4020-b553-b88951266cdb" containerName="discovery" containerID="cri-o://1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864" gracePeriod=30
Jul 16 21:54:11.406123 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.406094    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:11.463823 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.463802    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkxqr\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-kube-api-access-dkxqr\") pod \"535094a3-8021-4020-b553-b88951266cdb\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") "
Jul 16 21:54:11.463957 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.463834    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/535094a3-8021-4020-b553-b88951266cdb-local-certs\") pod \"535094a3-8021-4020-b553-b88951266cdb\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") "
Jul 16 21:54:11.463957 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.463861    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-kubeconfig\") pod \"535094a3-8021-4020-b553-b88951266cdb\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") "
Jul 16 21:54:11.463957 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.463890    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-istio-token\") pod \"535094a3-8021-4020-b553-b88951266cdb\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") "
Jul 16 21:54:11.463957 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.463928    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-csr-dns-cert\") pod \"535094a3-8021-4020-b553-b88951266cdb\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") "
Jul 16 21:54:11.464129 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.463964    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/535094a3-8021-4020-b553-b88951266cdb-istio-csr-ca-configmap\") pod \"535094a3-8021-4020-b553-b88951266cdb\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") "
Jul 16 21:54:11.464129 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.463993    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-cacerts\") pod \"535094a3-8021-4020-b553-b88951266cdb\" (UID: \"535094a3-8021-4020-b553-b88951266cdb\") "
Jul 16 21:54:11.470555 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.469860    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/535094a3-8021-4020-b553-b88951266cdb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "535094a3-8021-4020-b553-b88951266cdb" (UID: "535094a3-8021-4020-b553-b88951266cdb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:54:11.472197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.472146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/535094a3-8021-4020-b553-b88951266cdb-local-certs" (OuterVolumeSpecName: "local-certs") pod "535094a3-8021-4020-b553-b88951266cdb" (UID: "535094a3-8021-4020-b553-b88951266cdb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:54:11.472197 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.472109    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-kube-api-access-dkxqr" (OuterVolumeSpecName: "kube-api-access-dkxqr") pod "535094a3-8021-4020-b553-b88951266cdb" (UID: "535094a3-8021-4020-b553-b88951266cdb"). InnerVolumeSpecName "kube-api-access-dkxqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:54:11.473082 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.472462    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-cacerts" (OuterVolumeSpecName: "cacerts") pod "535094a3-8021-4020-b553-b88951266cdb" (UID: "535094a3-8021-4020-b553-b88951266cdb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:54:11.473082 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.472945    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-istio-token" (OuterVolumeSpecName: "istio-token") pod "535094a3-8021-4020-b553-b88951266cdb" (UID: "535094a3-8021-4020-b553-b88951266cdb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:54:11.473374 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.473353    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "535094a3-8021-4020-b553-b88951266cdb" (UID: "535094a3-8021-4020-b553-b88951266cdb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:54:11.474437 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.474413    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "535094a3-8021-4020-b553-b88951266cdb" (UID: "535094a3-8021-4020-b553-b88951266cdb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:54:11.565284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.565256    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/535094a3-8021-4020-b553-b88951266cdb-istio-csr-ca-configmap\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:11.565284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.565282    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-cacerts\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:11.565468 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.565292    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkxqr\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-kube-api-access-dkxqr\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:11.565468 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.565302    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/535094a3-8021-4020-b553-b88951266cdb-local-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:11.565468 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.565310    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-kubeconfig\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:11.565468 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.565318    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/535094a3-8021-4020-b553-b88951266cdb-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:11.565468 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.565326    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/535094a3-8021-4020-b553-b88951266cdb-istio-csr-dns-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:11.610786 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.610747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8" event={"ID":"0653006a-640b-4264-be65-9c922d3d7f3a","Type":"ContainerStarted","Data":"dbec74cf2176391498e0acfcf8dae7f53193918033aba3b019a889c8377646c3"}
Jul 16 21:54:11.610974 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.610858    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:11.612219 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.612196    2564 generic.go:356] "Generic (PLEG): container finished" podID="535094a3-8021-4020-b553-b88951266cdb" containerID="1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864" exitCode=0
Jul 16 21:54:11.612323 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.612222    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv" event={"ID":"535094a3-8021-4020-b553-b88951266cdb","Type":"ContainerDied","Data":"1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864"}
Jul 16 21:54:11.612323 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.612241    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"
Jul 16 21:54:11.612323 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.612266    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-khbnv" event={"ID":"535094a3-8021-4020-b553-b88951266cdb","Type":"ContainerDied","Data":"fab42dfb6ff358b4efcfeac7a74a6a8a78e3d432a02ea1017d67f841c95ee5f5"}
Jul 16 21:54:11.612323 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.612293    2564 scope.go:117] "RemoveContainer" containerID="1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864"
Jul 16 21:54:11.613982 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.613951    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5" event={"ID":"cd872a79-78f1-4c79-b884-6b2723da3867","Type":"ContainerStarted","Data":"9615777a60cda93cd22389f040b58a551bb2406a3159e3f0941de22bf8a2f26e"}
Jul 16 21:54:11.614102 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.614065    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:11.626905 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.626885    2564 scope.go:117] "RemoveContainer" containerID="1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864"
Jul 16 21:54:11.627235 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:54:11.627216    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864\": container with ID starting with 1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864 not found: ID does not exist" containerID="1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864"
Jul 16 21:54:11.627290 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.627244    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864"} err="failed to get container status \"1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864\": rpc error: code = NotFound desc = could not find container \"1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864\": container with ID starting with 1a8ae2853616fbd1f80bd2be17ce01a3803da0b58ccbed43e3d1adc2d4694864 not found: ID does not exist"
Jul 16 21:54:11.638384 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.638309    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8" podStartSLOduration=1.402587008 podStartE2EDuration="4.638296174s" totalImagesPullingTime="3.235709166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:11.637458137 +0000 UTC m=+318.026731935" watchObservedRunningTime="2026-07-16 21:54:11.638296174 +0000 UTC m=+318.027569960"
Jul 16 21:54:11.681693 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.681645    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5" podStartSLOduration=1.404200552 podStartE2EDuration="4.681625576s" totalImagesPullingTime="3.277425024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:11.681322636 +0000 UTC m=+318.070596442" watchObservedRunningTime="2026-07-16 21:54:11.681625576 +0000 UTC m=+318.070899362"
Jul 16 21:54:11.720370 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.720340    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"]
Jul 16 21:54:11.723525 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:11.723499    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-khbnv"]
Jul 16 21:54:12.125799 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:12.125766    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="535094a3-8021-4020-b553-b88951266cdb" path="/var/lib/kubelet/pods/535094a3-8021-4020-b553-b88951266cdb/volumes"
Jul 16 21:54:21.910279 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:21.910239    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:21.961766 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:21.961723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:42.621710 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:42.621681    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bvzc8"
Jul 16 21:54:42.624812 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:42.624793    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tlhk5"
Jul 16 21:54:54.956610 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:54.956573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:55.064990 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.064955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:55.074416 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.074390    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sk5gp"]
Jul 16 21:54:55.074708 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.074697    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="535094a3-8021-4020-b553-b88951266cdb" containerName="discovery"
Jul 16 21:54:55.074748 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.074710    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="535094a3-8021-4020-b553-b88951266cdb" containerName="discovery"
Jul 16 21:54:55.074789 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.074780    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="535094a3-8021-4020-b553-b88951266cdb" containerName="discovery"
Jul 16 21:54:55.078167 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.078142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp"
Jul 16 21:54:55.083397 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.083375    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 21:54:55.084106 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.084088    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xfrn6\""
Jul 16 21:54:55.084192 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.084095    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 21:54:55.094095 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.094067    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sk5gp"]
Jul 16 21:54:55.113158 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.113131    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2dcj\" (UniqueName: \"kubernetes.io/projected/fc3e8662-58ef-4622-8221-50b6c41dd714-kube-api-access-x2dcj\") pod \"kuadrant-operator-catalog-sk5gp\" (UID: \"fc3e8662-58ef-4622-8221-50b6c41dd714\") " pod="kuadrant-system/kuadrant-operator-catalog-sk5gp"
Jul 16 21:54:55.212299 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.212213    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:54:55.214224 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.214205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2dcj\" (UniqueName: \"kubernetes.io/projected/fc3e8662-58ef-4622-8221-50b6c41dd714-kube-api-access-x2dcj\") pod \"kuadrant-operator-catalog-sk5gp\" (UID: \"fc3e8662-58ef-4622-8221-50b6c41dd714\") " pod="kuadrant-system/kuadrant-operator-catalog-sk5gp"
Jul 16 21:54:55.244367 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.244339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2dcj\" (UniqueName: \"kubernetes.io/projected/fc3e8662-58ef-4622-8221-50b6c41dd714-kube-api-access-x2dcj\") pod \"kuadrant-operator-catalog-sk5gp\" (UID: \"fc3e8662-58ef-4622-8221-50b6c41dd714\") " pod="kuadrant-system/kuadrant-operator-catalog-sk5gp"
Jul 16 21:54:55.388461 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.388433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp"
Jul 16 21:54:55.417477 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.417449    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sk5gp"]
Jul 16 21:54:55.519244 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.519223    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sk5gp"]
Jul 16 21:54:55.521768 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:54:55.521740    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3e8662_58ef_4622_8221_50b6c41dd714.slice/crio-b9e1a4a23add81f2a61b9ba49ed8d8b7c84767e9ecabb70f4c95c46786019d15 WatchSource:0}: Error finding container b9e1a4a23add81f2a61b9ba49ed8d8b7c84767e9ecabb70f4c95c46786019d15: Status 404 returned error can't find the container with id b9e1a4a23add81f2a61b9ba49ed8d8b7c84767e9ecabb70f4c95c46786019d15
Jul 16 21:54:55.624404 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.624373    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fj82m"]
Jul 16 21:54:55.628006 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.627990    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:54:55.636894 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.636871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fj82m"]
Jul 16 21:54:55.718208 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.718173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-744jg\" (UniqueName: \"kubernetes.io/projected/e4cc2418-07cf-4b4e-a702-8c4006aeeca1-kube-api-access-744jg\") pod \"kuadrant-operator-catalog-fj82m\" (UID: \"e4cc2418-07cf-4b4e-a702-8c4006aeeca1\") " pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:54:55.811137 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.811066    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp" event={"ID":"fc3e8662-58ef-4622-8221-50b6c41dd714","Type":"ContainerStarted","Data":"b9e1a4a23add81f2a61b9ba49ed8d8b7c84767e9ecabb70f4c95c46786019d15"}
Jul 16 21:54:55.819457 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.819434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-744jg\" (UniqueName: \"kubernetes.io/projected/e4cc2418-07cf-4b4e-a702-8c4006aeeca1-kube-api-access-744jg\") pod \"kuadrant-operator-catalog-fj82m\" (UID: \"e4cc2418-07cf-4b4e-a702-8c4006aeeca1\") " pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:54:55.828068 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.828039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-744jg\" (UniqueName: \"kubernetes.io/projected/e4cc2418-07cf-4b4e-a702-8c4006aeeca1-kube-api-access-744jg\") pod \"kuadrant-operator-catalog-fj82m\" (UID: \"e4cc2418-07cf-4b4e-a702-8c4006aeeca1\") " pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:54:55.938430 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:55.938405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:54:56.070295 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:56.070268    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fj82m"]
Jul 16 21:54:56.072805 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:54:56.072777    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cc2418_07cf_4b4e_a702_8c4006aeeca1.slice/crio-d5d3f4fe6d016325e716bb3b30b9289c4f75fa3b86bbea3eae6eadd5ad3a4035 WatchSource:0}: Error finding container d5d3f4fe6d016325e716bb3b30b9289c4f75fa3b86bbea3eae6eadd5ad3a4035: Status 404 returned error can't find the container with id d5d3f4fe6d016325e716bb3b30b9289c4f75fa3b86bbea3eae6eadd5ad3a4035
Jul 16 21:54:56.817981 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:56.817938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fj82m" event={"ID":"e4cc2418-07cf-4b4e-a702-8c4006aeeca1","Type":"ContainerStarted","Data":"d5d3f4fe6d016325e716bb3b30b9289c4f75fa3b86bbea3eae6eadd5ad3a4035"}
Jul 16 21:54:57.823438 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:57.823334    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fj82m" event={"ID":"e4cc2418-07cf-4b4e-a702-8c4006aeeca1","Type":"ContainerStarted","Data":"c745de35e9f6274b3e41d6e167729e2dc93fd3534483b0e15c6e6ab18252d664"}
Jul 16 21:54:57.824753 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:57.824728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp" event={"ID":"fc3e8662-58ef-4622-8221-50b6c41dd714","Type":"ContainerStarted","Data":"e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519"}
Jul 16 21:54:57.824895 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:57.824800    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp" podUID="fc3e8662-58ef-4622-8221-50b6c41dd714" containerName="registry-server" containerID="cri-o://e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519" gracePeriod=2
Jul 16 21:54:57.839876 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:57.839829    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fj82m" podStartSLOduration=1.335471074 podStartE2EDuration="2.839814581s" totalImagesPullingTime="1.504343507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:57.838750064 +0000 UTC m=+364.228023850" watchObservedRunningTime="2026-07-16 21:54:57.839814581 +0000 UTC m=+364.229088367"
Jul 16 21:54:57.855090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:57.855049    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp" podStartSLOduration=0.800044812 podStartE2EDuration="2.855035816s" totalImagesPullingTime="2.054991004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:57.853782579 +0000 UTC m=+364.243056364" watchObservedRunningTime="2026-07-16 21:54:57.855035816 +0000 UTC m=+364.244309602"
Jul 16 21:54:58.062151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.062127    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp"
Jul 16 21:54:58.138194 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.138126    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2dcj\" (UniqueName: \"kubernetes.io/projected/fc3e8662-58ef-4622-8221-50b6c41dd714-kube-api-access-x2dcj\") pod \"fc3e8662-58ef-4622-8221-50b6c41dd714\" (UID: \"fc3e8662-58ef-4622-8221-50b6c41dd714\") "
Jul 16 21:54:58.140357 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.140328    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3e8662-58ef-4622-8221-50b6c41dd714-kube-api-access-x2dcj" (OuterVolumeSpecName: "kube-api-access-x2dcj") pod "fc3e8662-58ef-4622-8221-50b6c41dd714" (UID: "fc3e8662-58ef-4622-8221-50b6c41dd714"). InnerVolumeSpecName "kube-api-access-x2dcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:54:58.239076 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.239042    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2dcj\" (UniqueName: \"kubernetes.io/projected/fc3e8662-58ef-4622-8221-50b6c41dd714-kube-api-access-x2dcj\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:54:58.830079 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.830037    2564 generic.go:356] "Generic (PLEG): container finished" podID="fc3e8662-58ef-4622-8221-50b6c41dd714" containerID="e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519" exitCode=0
Jul 16 21:54:58.830519 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.830091    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp"
Jul 16 21:54:58.830519 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.830118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp" event={"ID":"fc3e8662-58ef-4622-8221-50b6c41dd714","Type":"ContainerDied","Data":"e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519"}
Jul 16 21:54:58.830519 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.830166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sk5gp" event={"ID":"fc3e8662-58ef-4622-8221-50b6c41dd714","Type":"ContainerDied","Data":"b9e1a4a23add81f2a61b9ba49ed8d8b7c84767e9ecabb70f4c95c46786019d15"}
Jul 16 21:54:58.830519 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.830186    2564 scope.go:117] "RemoveContainer" containerID="e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519"
Jul 16 21:54:58.839669 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.839649    2564 scope.go:117] "RemoveContainer" containerID="e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519"
Jul 16 21:54:58.839959 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:54:58.839933    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519\": container with ID starting with e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519 not found: ID does not exist" containerID="e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519"
Jul 16 21:54:58.840039 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.839966    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519"} err="failed to get container status \"e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519\": rpc error: code = NotFound desc = could not find container \"e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519\": container with ID starting with e0acbb651d7302b48f3aa56052659fb43a73f271d6d790d386308187ee1ed519 not found: ID does not exist"
Jul 16 21:54:58.851690 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.851671    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sk5gp"]
Jul 16 21:54:58.856183 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:58.856140    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sk5gp"]
Jul 16 21:54:59.979115 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:54:59.979082    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:00.079206 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:00.079174    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:00.124342 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:00.124316    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc3e8662-58ef-4622-8221-50b6c41dd714" path="/var/lib/kubelet/pods/fc3e8662-58ef-4622-8221-50b6c41dd714/volumes"
Jul 16 21:55:01.827213 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:01.827172    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:05.938931 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:05.938880    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:55:05.939366 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:05.938966    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:55:05.961493 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:05.961470    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:55:06.889499 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:06.889473    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-fj82m"
Jul 16 21:55:19.389609 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:19.389573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:22.011902 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.011869    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"]
Jul 16 21:55:22.012274 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.012196    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc3e8662-58ef-4622-8221-50b6c41dd714" containerName="registry-server"
Jul 16 21:55:22.012274 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.012207    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3e8662-58ef-4622-8221-50b6c41dd714" containerName="registry-server"
Jul 16 21:55:22.012274 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.012257    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc3e8662-58ef-4622-8221-50b6c41dd714" containerName="registry-server"
Jul 16 21:55:22.015079 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.015064    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"
Jul 16 21:55:22.019841 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:22.019812    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-operator-controller-manager-dockercfg-c87p2\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c87p2\"" type="*v1.Secret"
Jul 16 21:55:22.019841 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.019838    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 16 21:55:22.059367 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.059343    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"]
Jul 16 21:55:22.131532 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.131508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ltd\" (UniqueName: \"kubernetes.io/projected/49284255-ad2e-40b5-9ab4-916e2c1c69dc-kube-api-access-n2ltd\") pod \"dns-operator-controller-manager-864dd78c4c-8sznl\" (UID: \"49284255-ad2e-40b5-9ab4-916e2c1c69dc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"
Jul 16 21:55:22.232383 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.232345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ltd\" (UniqueName: \"kubernetes.io/projected/49284255-ad2e-40b5-9ab4-916e2c1c69dc-kube-api-access-n2ltd\") pod \"dns-operator-controller-manager-864dd78c4c-8sznl\" (UID: \"49284255-ad2e-40b5-9ab4-916e2c1c69dc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"
Jul 16 21:55:22.262504 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:22.262417    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ltd\" (UniqueName: \"kubernetes.io/projected/49284255-ad2e-40b5-9ab4-916e2c1c69dc-kube-api-access-n2ltd\") pod \"dns-operator-controller-manager-864dd78c4c-8sznl\" (UID: \"49284255-ad2e-40b5-9ab4-916e2c1c69dc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"
Jul 16 21:55:23.221101 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:23.221067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:23.324778 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:23.324734    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 16 21:55:23.324961 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:23.324807    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"
Jul 16 21:55:23.496361 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:23.496336    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"]
Jul 16 21:55:23.497880 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:55:23.497856    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49284255_ad2e_40b5_9ab4_916e2c1c69dc.slice/crio-595a3c5103ef9142af812bcbb9d19dc311e51f5c72fd014fa08621a7bc4c1ad8 WatchSource:0}: Error finding container 595a3c5103ef9142af812bcbb9d19dc311e51f5c72fd014fa08621a7bc4c1ad8: Status 404 returned error can't find the container with id 595a3c5103ef9142af812bcbb9d19dc311e51f5c72fd014fa08621a7bc4c1ad8
Jul 16 21:55:23.531444 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:23.531420    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c87p2\""
Jul 16 21:55:23.939865 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:23.939831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl" event={"ID":"49284255-ad2e-40b5-9ab4-916e2c1c69dc","Type":"ContainerStarted","Data":"595a3c5103ef9142af812bcbb9d19dc311e51f5c72fd014fa08621a7bc4c1ad8"}
Jul 16 21:55:25.952071 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:25.952039    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl" event={"ID":"49284255-ad2e-40b5-9ab4-916e2c1c69dc","Type":"ContainerStarted","Data":"71a314c3a76cc043e7a1c7cdfe67506367650e7bc16a457c5eb175a87ef6f84a"}
Jul 16 21:55:25.952444 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:25.952147    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"
Jul 16 21:55:25.975826 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:25.975796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:26.017451 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:26.017399    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl" podStartSLOduration=3.035521627 podStartE2EDuration="5.017384895s" totalImagesPullingTime="1.981863268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:26.016061822 +0000 UTC m=+392.405335609" watchObservedRunningTime="2026-07-16 21:55:26.017384895 +0000 UTC m=+392.406658680"
Jul 16 21:55:27.224342 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:27.224314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:28.533895 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:28.533858    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:28.881027 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:28.880905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:29.509453 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.509417    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"]
Jul 16 21:55:29.512733 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.512715    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:29.519255 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.518903    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jv95z\""
Jul 16 21:55:29.590858 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.590828    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"]
Jul 16 21:55:29.603365 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.603334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d3fa089-7f36-4550-829d-b7bf95a07029-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:29.603510 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.603394    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69ns6\" (UniqueName: \"kubernetes.io/projected/2d3fa089-7f36-4550-829d-b7bf95a07029-kube-api-access-69ns6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:29.622403 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.622370    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:29.704324 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.704295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d3fa089-7f36-4550-829d-b7bf95a07029-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:29.704470 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.704334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69ns6\" (UniqueName: \"kubernetes.io/projected/2d3fa089-7f36-4550-829d-b7bf95a07029-kube-api-access-69ns6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:29.704679 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.704659    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d3fa089-7f36-4550-829d-b7bf95a07029-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:29.735152 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.735124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69ns6\" (UniqueName: \"kubernetes.io/projected/2d3fa089-7f36-4550-829d-b7bf95a07029-kube-api-access-69ns6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:29.822653 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:29.822581    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:30.069587 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:30.069548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"]
Jul 16 21:55:30.074301 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:55:30.074238    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3fa089_7f36_4550_829d_b7bf95a07029.slice/crio-1ff153a7b386f9ec22ab82cdaf62187569d55864dba2ae99ebf911594395e9ff WatchSource:0}: Error finding container 1ff153a7b386f9ec22ab82cdaf62187569d55864dba2ae99ebf911594395e9ff: Status 404 returned error can't find the container with id 1ff153a7b386f9ec22ab82cdaf62187569d55864dba2ae99ebf911594395e9ff
Jul 16 21:55:30.975252 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:30.975218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z" event={"ID":"2d3fa089-7f36-4550-829d-b7bf95a07029","Type":"ContainerStarted","Data":"1ff153a7b386f9ec22ab82cdaf62187569d55864dba2ae99ebf911594395e9ff"}
Jul 16 21:55:31.718256 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:31.718222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:31.915582 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:31.915544    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"]
Jul 16 21:55:31.918831 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:31.918809    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"
Jul 16 21:55:31.932396 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:31.932377    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8nhcs\""
Jul 16 21:55:31.952204 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:31.952179    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"]
Jul 16 21:55:32.025669 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:32.025596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9jq8\" (UniqueName: \"kubernetes.io/projected/a27f3c30-2fe1-4d31-9b35-6aea679a1e00-kube-api-access-m9jq8\") pod \"authorino-operator-75cc8d5cb7-9n45m\" (UID: \"a27f3c30-2fe1-4d31-9b35-6aea679a1e00\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"
Jul 16 21:55:32.126427 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:32.126402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9jq8\" (UniqueName: \"kubernetes.io/projected/a27f3c30-2fe1-4d31-9b35-6aea679a1e00-kube-api-access-m9jq8\") pod \"authorino-operator-75cc8d5cb7-9n45m\" (UID: \"a27f3c30-2fe1-4d31-9b35-6aea679a1e00\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"
Jul 16 21:55:32.151877 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:32.151844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9jq8\" (UniqueName: \"kubernetes.io/projected/a27f3c30-2fe1-4d31-9b35-6aea679a1e00-kube-api-access-m9jq8\") pod \"authorino-operator-75cc8d5cb7-9n45m\" (UID: \"a27f3c30-2fe1-4d31-9b35-6aea679a1e00\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"
Jul 16 21:55:32.228577 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:32.228544    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"
Jul 16 21:55:32.437611 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:32.437584    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"]
Jul 16 21:55:32.440893 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:55:32.440868    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27f3c30_2fe1_4d31_9b35_6aea679a1e00.slice/crio-dff8d2f2be127b293e73a3ababa2c467305fe064e5fa32f50475506bc0b53cb6 WatchSource:0}: Error finding container dff8d2f2be127b293e73a3ababa2c467305fe064e5fa32f50475506bc0b53cb6: Status 404 returned error can't find the container with id dff8d2f2be127b293e73a3ababa2c467305fe064e5fa32f50475506bc0b53cb6
Jul 16 21:55:32.991273 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:32.991162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m" event={"ID":"a27f3c30-2fe1-4d31-9b35-6aea679a1e00","Type":"ContainerStarted","Data":"dff8d2f2be127b293e73a3ababa2c467305fe064e5fa32f50475506bc0b53cb6"}
Jul 16 21:55:33.602324 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:33.602282    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:33.668191 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:33.668161    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"]
Jul 16 21:55:33.673246 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:33.673222    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:33.678539 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:33.678502    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:33.688809 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:33.688775    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"limitador-operator-controller-manager-dockercfg-8c8g8\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8c8g8\"" type="*v1.Secret"
Jul 16 21:55:33.729471 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:33.729365    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"]
Jul 16 21:55:33.840231 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:33.840196    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8789v\" (UniqueName: \"kubernetes.io/projected/06d08951-3931-4f86-9866-4d017d169fa3-kube-api-access-8789v\") pod \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" (UID: \"06d08951-3931-4f86-9866-4d017d169fa3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:33.941475 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:33.941430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8789v\" (UniqueName: \"kubernetes.io/projected/06d08951-3931-4f86-9866-4d017d169fa3-kube-api-access-8789v\") pod \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" (UID: \"06d08951-3931-4f86-9866-4d017d169fa3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:33.957735 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:33.957704    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8789v\" (UniqueName: \"kubernetes.io/projected/06d08951-3931-4f86-9866-4d017d169fa3-kube-api-access-8789v\") pod \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" (UID: \"06d08951-3931-4f86-9866-4d017d169fa3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:34.909029 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:34.908894    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:34.988159 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:34.988117    2564 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 16 21:55:34.988339 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:34.988233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:35.028391 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:35.028364    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8c8g8\""
Jul 16 21:55:35.423412 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:35.423379    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:35.990099 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:35.990068    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"]
Jul 16 21:55:35.998257 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:55:35.998230    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d08951_3931_4f86_9866_4d017d169fa3.slice/crio-0b19491f594c5f583fac6aba48d8fef9247e72ba34c68336c6132a40cbbdc27e WatchSource:0}: Error finding container 0b19491f594c5f583fac6aba48d8fef9247e72ba34c68336c6132a40cbbdc27e: Status 404 returned error can't find the container with id 0b19491f594c5f583fac6aba48d8fef9247e72ba34c68336c6132a40cbbdc27e
Jul 16 21:55:36.011200 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.011173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh" event={"ID":"06d08951-3931-4f86-9866-4d017d169fa3","Type":"ContainerStarted","Data":"0b19491f594c5f583fac6aba48d8fef9247e72ba34c68336c6132a40cbbdc27e"}
Jul 16 21:55:36.012745 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.012722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m" event={"ID":"a27f3c30-2fe1-4d31-9b35-6aea679a1e00","Type":"ContainerStarted","Data":"7e8846e8ca3e343c04325a9c1320faba6049c224c4e10ec623db201bf4c56018"}
Jul 16 21:55:36.012921 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.012889    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"
Jul 16 21:55:36.014621 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.014601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z" event={"ID":"2d3fa089-7f36-4550-829d-b7bf95a07029","Type":"ContainerStarted","Data":"70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72"}
Jul 16 21:55:36.014707 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.014692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:36.047793 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.047697    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m" podStartSLOduration=1.693676005 podStartE2EDuration="5.047678655s" totalImagesPullingTime="3.35400265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:36.044793027 +0000 UTC m=+402.434066812" watchObservedRunningTime="2026-07-16 21:55:36.047678655 +0000 UTC m=+402.436952441"
Jul 16 21:55:36.077012 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.076952    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z" podStartSLOduration=1.2869347709999999 podStartE2EDuration="7.076936877s" totalImagesPullingTime="5.790002106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:36.073807414 +0000 UTC m=+402.463081213" watchObservedRunningTime="2026-07-16 21:55:36.076936877 +0000 UTC m=+402.466210662"
Jul 16 21:55:36.959162 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:36.959134    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8sznl"
Jul 16 21:55:38.027805 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:38.027766    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh" event={"ID":"06d08951-3931-4f86-9866-4d017d169fa3","Type":"ContainerStarted","Data":"9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb"}
Jul 16 21:55:38.028233 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:38.027903    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:38.111475 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:38.108460    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh" podStartSLOduration=3.9107261810000002 podStartE2EDuration="5.108441195s" totalImagesPullingTime="1.197715014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:38.106227697 +0000 UTC m=+404.495501485" watchObservedRunningTime="2026-07-16 21:55:38.108441195 +0000 UTC m=+404.497714985"
Jul 16 21:55:39.090113 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:39.090078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:39.612230 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:39.612198    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:47.023326 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:47.023296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:47.023759 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:47.023467    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9n45m"
Jul 16 21:55:48.008159 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:48.008107    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:48.529546 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:48.529518    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:48.867294 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:48.867191    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:49.035276 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:49.035248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:50.473551 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.473519    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"]
Jul 16 21:55:50.474090 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.473720    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" containerName="manager" containerID="cri-o://70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72" gracePeriod=2
Jul 16 21:55:50.486151 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.486094    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"]
Jul 16 21:55:50.519934 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.519898    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"]
Jul 16 21:55:50.520136 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.520115    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh" podUID="06d08951-3931-4f86-9866-4d017d169fa3" containerName="manager" containerID="cri-o://9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb" gracePeriod=2
Jul 16 21:55:50.542154 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.542121    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"]
Jul 16 21:55:50.550795 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:50.550760    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:50.606022 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.605987    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"]
Jul 16 21:55:50.606344 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.606329    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d08951-3931-4f86-9866-4d017d169fa3" containerName="manager"
Jul 16 21:55:50.606395 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.606348    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d08951-3931-4f86-9866-4d017d169fa3" containerName="manager"
Jul 16 21:55:50.606395 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.606370    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" containerName="manager"
Jul 16 21:55:50.606395 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.606377    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" containerName="manager"
Jul 16 21:55:50.606486 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.606435    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d08951-3931-4f86-9866-4d017d169fa3" containerName="manager"
Jul 16 21:55:50.606486 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.606445    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" containerName="manager"
Jul 16 21:55:50.609299 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.609284    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.655614 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.655573    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"]
Jul 16 21:55:50.658685 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:50.658652    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:50.660889 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.660002    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"
Jul 16 21:55:50.661106 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:50.661076    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:50.667798 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.667768    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqfnj\" (UniqueName: \"kubernetes.io/projected/f360a550-9cbf-4708-a2ed-aa4338df0be8-kube-api-access-lqfnj\") pod \"limitador-operator-controller-manager-859d7b78bd-kdkrg\" (UID: \"f360a550-9cbf-4708-a2ed-aa4338df0be8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"
Jul 16 21:55:50.667968 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.667944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkfgb\" (UniqueName: \"kubernetes.io/projected/83835db4-173e-4734-8faf-afa607343ee6-kube-api-access-lkfgb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-278cq\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.668087 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.668049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83835db4-173e-4734-8faf-afa607343ee6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-278cq\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.717351 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.717318    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"]
Jul 16 21:55:50.756554 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:50.756515    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:50.769116 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.769085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkfgb\" (UniqueName: \"kubernetes.io/projected/83835db4-173e-4734-8faf-afa607343ee6-kube-api-access-lkfgb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-278cq\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.769276 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.769140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83835db4-173e-4734-8faf-afa607343ee6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-278cq\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.769276 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.769200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqfnj\" (UniqueName: \"kubernetes.io/projected/f360a550-9cbf-4708-a2ed-aa4338df0be8-kube-api-access-lqfnj\") pod \"limitador-operator-controller-manager-859d7b78bd-kdkrg\" (UID: \"f360a550-9cbf-4708-a2ed-aa4338df0be8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"
Jul 16 21:55:50.769554 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.769529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83835db4-173e-4734-8faf-afa607343ee6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-278cq\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.807787 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.807761    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"]
Jul 16 21:55:50.811578 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.811553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqfnj\" (UniqueName: \"kubernetes.io/projected/f360a550-9cbf-4708-a2ed-aa4338df0be8-kube-api-access-lqfnj\") pod \"limitador-operator-controller-manager-859d7b78bd-kdkrg\" (UID: \"f360a550-9cbf-4708-a2ed-aa4338df0be8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"
Jul 16 21:55:50.813031 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.813016    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:50.816110 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.816089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkfgb\" (UniqueName: \"kubernetes.io/projected/83835db4-173e-4734-8faf-afa607343ee6-kube-api-access-lkfgb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-278cq\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.827838 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.827820    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:50.870165 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.870139    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d3fa089-7f36-4550-829d-b7bf95a07029-extensions-socket-volume\") pod \"2d3fa089-7f36-4550-829d-b7bf95a07029\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") "
Jul 16 21:55:50.870260 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.870195    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8789v\" (UniqueName: \"kubernetes.io/projected/06d08951-3931-4f86-9866-4d017d169fa3-kube-api-access-8789v\") pod \"06d08951-3931-4f86-9866-4d017d169fa3\" (UID: \"06d08951-3931-4f86-9866-4d017d169fa3\") "
Jul 16 21:55:50.870260 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.870233    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69ns6\" (UniqueName: \"kubernetes.io/projected/2d3fa089-7f36-4550-829d-b7bf95a07029-kube-api-access-69ns6\") pod \"2d3fa089-7f36-4550-829d-b7bf95a07029\" (UID: \"2d3fa089-7f36-4550-829d-b7bf95a07029\") "
Jul 16 21:55:50.870594 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.870570    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d3fa089-7f36-4550-829d-b7bf95a07029-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2d3fa089-7f36-4550-829d-b7bf95a07029" (UID: "2d3fa089-7f36-4550-829d-b7bf95a07029"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:55:50.872284 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.872253    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d08951-3931-4f86-9866-4d017d169fa3-kube-api-access-8789v" (OuterVolumeSpecName: "kube-api-access-8789v") pod "06d08951-3931-4f86-9866-4d017d169fa3" (UID: "06d08951-3931-4f86-9866-4d017d169fa3"). InnerVolumeSpecName "kube-api-access-8789v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:55:50.872363 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.872292    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3fa089-7f36-4550-829d-b7bf95a07029-kube-api-access-69ns6" (OuterVolumeSpecName: "kube-api-access-69ns6") pod "2d3fa089-7f36-4550-829d-b7bf95a07029" (UID: "2d3fa089-7f36-4550-829d-b7bf95a07029"). InnerVolumeSpecName "kube-api-access-69ns6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:55:50.883404 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:50.883372    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:50.889684 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:50.889655    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:50.892593 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:50.892572    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:50.918775 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.918754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:50.970856 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.970825    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69ns6\" (UniqueName: \"kubernetes.io/projected/2d3fa089-7f36-4550-829d-b7bf95a07029-kube-api-access-69ns6\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:55:50.971005 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.970858    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d3fa089-7f36-4550-829d-b7bf95a07029-extensions-socket-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:55:50.971005 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.970872    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8789v\" (UniqueName: \"kubernetes.io/projected/06d08951-3931-4f86-9866-4d017d169fa3-kube-api-access-8789v\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:55:50.971815 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:50.971796    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"
Jul 16 21:55:51.087703 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.087665    2564 generic.go:356] "Generic (PLEG): container finished" podID="06d08951-3931-4f86-9866-4d017d169fa3" containerID="9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb" exitCode=0
Jul 16 21:55:51.087860 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.087770    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:51.087860 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.087811    2564 scope.go:117] "RemoveContainer" containerID="9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb"
Jul 16 21:55:51.091848 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.089485    2564 generic.go:356] "Generic (PLEG): container finished" podID="2d3fa089-7f36-4550-829d-b7bf95a07029" containerID="70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72" exitCode=0
Jul 16 21:55:51.091848 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.089569    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:51.092392 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.092253    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.099475    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.100810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.101316    2564 scope.go:117] "RemoveContainer" containerID="9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb"
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.101690    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb\": container with ID starting with 9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb not found: ID does not exist" containerID="9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb"
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.101734    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb"} err="failed to get container status \"9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb\": rpc error: code = NotFound desc = could not find container \"9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb\": container with ID starting with 9d832140eeaa929088afa70a229585e5f6b608f05399623a6750392ea29047cb not found: ID does not exist"
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.101756    2564 scope.go:117] "RemoveContainer" containerID="70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72"
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.102609    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:51.103522 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.102674    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"]
Jul 16 21:55:51.104435 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:55:51.104408    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83835db4_173e_4734_8faf_afa607343ee6.slice/crio-3688b63cb1010e7afd3dfba27bd4a8a0bedbbf746072501f9cd18f053a5dc203 WatchSource:0}: Error finding container 3688b63cb1010e7afd3dfba27bd4a8a0bedbbf746072501f9cd18f053a5dc203: Status 404 returned error can't find the container with id 3688b63cb1010e7afd3dfba27bd4a8a0bedbbf746072501f9cd18f053a5dc203
Jul 16 21:55:51.105569 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.105534    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:51.111462 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.111446    2564 scope.go:117] "RemoveContainer" containerID="70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72"
Jul 16 21:55:51.111703 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.111687    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72\": container with ID starting with 70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72 not found: ID does not exist" containerID="70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72"
Jul 16 21:55:51.111747 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.111711    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72"} err="failed to get container status \"70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72\": rpc error: code = NotFound desc = could not find container \"70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72\": container with ID starting with 70bfcb39095991654f763029b34d1c916e80357daacf9c3cedf7323bb300cf72 not found: ID does not exist"
Jul 16 21:55:51.126323 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.126300    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:51.128105 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:51.128078    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:51.165297 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.165272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"]
Jul 16 21:55:51.168882 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:55:51.168858    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf360a550_9cbf_4708_a2ed_aa4338df0be8.slice/crio-c258683100f4f8c52015377cb47dbef72f4f1a0c95b610f11057667411af6c88 WatchSource:0}: Error finding container c258683100f4f8c52015377cb47dbef72f4f1a0c95b610f11057667411af6c88: Status 404 returned error can't find the container with id c258683100f4f8c52015377cb47dbef72f4f1a0c95b610f11057667411af6c88
Jul 16 21:55:51.663877 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:51.663840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:52.096991 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.096947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg" event={"ID":"f360a550-9cbf-4708-a2ed-aa4338df0be8","Type":"ContainerStarted","Data":"e98fb04bcd27143d82834494a23ef003bfd3ddd0d5dcbb8dc285ed4646b5c28f"}
Jul 16 21:55:52.097175 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.097014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg" event={"ID":"f360a550-9cbf-4708-a2ed-aa4338df0be8","Type":"ContainerStarted","Data":"c258683100f4f8c52015377cb47dbef72f4f1a0c95b610f11057667411af6c88"}
Jul 16 21:55:52.097175 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.097056    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"
Jul 16 21:55:52.099394 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.099371    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq" event={"ID":"83835db4-173e-4734-8faf-afa607343ee6","Type":"ContainerStarted","Data":"5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32"}
Jul 16 21:55:52.099506 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.099406    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq" event={"ID":"83835db4-173e-4734-8faf-afa607343ee6","Type":"ContainerStarted","Data":"3688b63cb1010e7afd3dfba27bd4a8a0bedbbf746072501f9cd18f053a5dc203"}
Jul 16 21:55:52.099506 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.099440    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:55:52.124970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.124933    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06d08951-3931-4f86-9866-4d017d169fa3" path="/var/lib/kubelet/pods/06d08951-3931-4f86-9866-4d017d169fa3/volumes"
Jul 16 21:55:52.125295 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.125281    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" path="/var/lib/kubelet/pods/2d3fa089-7f36-4550-829d-b7bf95a07029/volumes"
Jul 16 21:55:52.153011 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:52.152954    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg" podStartSLOduration=2.152939457 podStartE2EDuration="2.152939457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:52.152110895 +0000 UTC m=+418.541384681" watchObservedRunningTime="2026-07-16 21:55:52.152939457 +0000 UTC m=+418.542213243"
Jul 16 21:55:52.154343 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:52.154312    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dd4jh\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="06d08951-3931-4f86-9866-4d017d169fa3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dd4jh"
Jul 16 21:55:52.160274 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:55:52.160253    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zsb9z\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="2d3fa089-7f36-4550-829d-b7bf95a07029" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zsb9z"
Jul 16 21:55:53.891021 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:53.890982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:54.160309 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:54.160211    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:54.683650 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:54.683610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:54.865044 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:54.864997    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:56.447378 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.447331    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq" podStartSLOduration=6.447314037 podStartE2EDuration="6.447314037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:52.20013791 +0000 UTC m=+418.589411696" watchObservedRunningTime="2026-07-16 21:55:56.447314037 +0000 UTC m=+422.836587822"
Jul 16 21:55:56.447831 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.447816    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"]
Jul 16 21:55:56.454130 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.454113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.509373 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.509343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fafd3-e4c4-4eeb-9f83-36656176b344-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lbxmw\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.509500 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.509392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm4xn\" (UniqueName: \"kubernetes.io/projected/703fafd3-e4c4-4eeb-9f83-36656176b344-kube-api-access-tm4xn\") pod \"kuadrant-operator-controller-manager-86878c769f-lbxmw\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.551341 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.551312    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"]
Jul 16 21:55:56.610115 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.610086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fafd3-e4c4-4eeb-9f83-36656176b344-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lbxmw\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.610259 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.610140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm4xn\" (UniqueName: \"kubernetes.io/projected/703fafd3-e4c4-4eeb-9f83-36656176b344-kube-api-access-tm4xn\") pod \"kuadrant-operator-controller-manager-86878c769f-lbxmw\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.610431 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.610414    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fafd3-e4c4-4eeb-9f83-36656176b344-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lbxmw\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.623463 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.623435    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm4xn\" (UniqueName: \"kubernetes.io/projected/703fafd3-e4c4-4eeb-9f83-36656176b344-kube-api-access-tm4xn\") pod \"kuadrant-operator-controller-manager-86878c769f-lbxmw\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.764294 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.764227    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:56.960657 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:56.960631    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"]
Jul 16 21:55:56.962863 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:55:56.962843    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703fafd3_e4c4_4eeb_9f83_36656176b344.slice/crio-8c061020d1d15a95eaccb197c3209e1f5c8ea7e9e0e8f5a8917af19d7a3710d3 WatchSource:0}: Error finding container 8c061020d1d15a95eaccb197c3209e1f5c8ea7e9e0e8f5a8917af19d7a3710d3: Status 404 returned error can't find the container with id 8c061020d1d15a95eaccb197c3209e1f5c8ea7e9e0e8f5a8917af19d7a3710d3
Jul 16 21:55:57.124580 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:57.124550    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw" event={"ID":"703fafd3-e4c4-4eeb-9f83-36656176b344","Type":"ContainerStarted","Data":"fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1"}
Jul 16 21:55:57.124735 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:57.124600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw" event={"ID":"703fafd3-e4c4-4eeb-9f83-36656176b344","Type":"ContainerStarted","Data":"8c061020d1d15a95eaccb197c3209e1f5c8ea7e9e0e8f5a8917af19d7a3710d3"}
Jul 16 21:55:57.124804 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:57.124727    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:55:57.566959 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:57.566924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:55:57.961126 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:55:57.961086    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:03.106997 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:03.106966    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kdkrg"
Jul 16 21:56:03.107347 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:03.107041    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:56:03.155785 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:03.155742    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw" podStartSLOduration=7.155727421 podStartE2EDuration="7.155727421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:57.186369261 +0000 UTC m=+423.575643048" watchObservedRunningTime="2026-07-16 21:56:03.155727421 +0000 UTC m=+429.545001206"
Jul 16 21:56:05.936406 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:05.936365    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:06.100525 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:06.100494    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:06.648893 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:06.648851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:06.852989 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:06.852898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:07.205863 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:07.205828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:08.132175 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:08.132146    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 21:56:08.259327 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:08.259294    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"]
Jul 16 21:56:08.259711 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:08.259493    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq" podUID="83835db4-173e-4734-8faf-afa607343ee6" containerName="manager" containerID="cri-o://5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32" gracePeriod=10
Jul 16 21:56:08.945183 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:08.945125    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:56:09.004538 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.004512    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkfgb\" (UniqueName: \"kubernetes.io/projected/83835db4-173e-4734-8faf-afa607343ee6-kube-api-access-lkfgb\") pod \"83835db4-173e-4734-8faf-afa607343ee6\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") "
Jul 16 21:56:09.004701 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.004576    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83835db4-173e-4734-8faf-afa607343ee6-extensions-socket-volume\") pod \"83835db4-173e-4734-8faf-afa607343ee6\" (UID: \"83835db4-173e-4734-8faf-afa607343ee6\") "
Jul 16 21:56:09.004970 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.004944    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83835db4-173e-4734-8faf-afa607343ee6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "83835db4-173e-4734-8faf-afa607343ee6" (UID: "83835db4-173e-4734-8faf-afa607343ee6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:56:09.006688 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.006657    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83835db4-173e-4734-8faf-afa607343ee6-kube-api-access-lkfgb" (OuterVolumeSpecName: "kube-api-access-lkfgb") pod "83835db4-173e-4734-8faf-afa607343ee6" (UID: "83835db4-173e-4734-8faf-afa607343ee6"). InnerVolumeSpecName "kube-api-access-lkfgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:56:09.105089 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.105058    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkfgb\" (UniqueName: \"kubernetes.io/projected/83835db4-173e-4734-8faf-afa607343ee6-kube-api-access-lkfgb\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:56:09.105089 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.105083    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83835db4-173e-4734-8faf-afa607343ee6-extensions-socket-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:56:09.179716 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.179683    2564 generic.go:356] "Generic (PLEG): container finished" podID="83835db4-173e-4734-8faf-afa607343ee6" containerID="5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32" exitCode=0
Jul 16 21:56:09.179867 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.179727    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"
Jul 16 21:56:09.179867 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.179726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq" event={"ID":"83835db4-173e-4734-8faf-afa607343ee6","Type":"ContainerDied","Data":"5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32"}
Jul 16 21:56:09.179867 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.179791    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq" event={"ID":"83835db4-173e-4734-8faf-afa607343ee6","Type":"ContainerDied","Data":"3688b63cb1010e7afd3dfba27bd4a8a0bedbbf746072501f9cd18f053a5dc203"}
Jul 16 21:56:09.179867 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.179824    2564 scope.go:117] "RemoveContainer" containerID="5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32"
Jul 16 21:56:09.188963 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.188947    2564 scope.go:117] "RemoveContainer" containerID="5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32"
Jul 16 21:56:09.189237 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:56:09.189217    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32\": container with ID starting with 5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32 not found: ID does not exist" containerID="5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32"
Jul 16 21:56:09.189304 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.189244    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32"} err="failed to get container status \"5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32\": rpc error: code = NotFound desc = could not find container \"5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32\": container with ID starting with 5a1827b1cd8b884f436aec3b75d49e615ed6dedf2fbbe967894226670fe2de32 not found: ID does not exist"
Jul 16 21:56:09.223814 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.223752    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"]
Jul 16 21:56:09.228560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:09.228540    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-278cq"]
Jul 16 21:56:10.124651 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:10.124606    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83835db4-173e-4734-8faf-afa607343ee6" path="/var/lib/kubelet/pods/83835db4-173e-4734-8faf-afa607343ee6/volumes"
Jul 16 21:56:10.852479 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:10.852443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:11.131554 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:11.131483    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:15.774266 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:15.774164    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:15.972619 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:15.972582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:19.995120 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:19.995082    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:20.141112 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:20.141077    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:25.260381 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:25.260347    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:25.957580 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:25.957532    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:27.726804 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.726770    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"]
Jul 16 21:56:27.727203 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.727113    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83835db4-173e-4734-8faf-afa607343ee6" containerName="manager"
Jul 16 21:56:27.727203 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.727124    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="83835db4-173e-4734-8faf-afa607343ee6" containerName="manager"
Jul 16 21:56:27.727203 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.727184    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="83835db4-173e-4734-8faf-afa607343ee6" containerName="manager"
Jul 16 21:56:27.730379 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.730354    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:27.730708 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.730673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:27.733230 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.733208    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cbks6\""
Jul 16 21:56:27.733342 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.733328    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 21:56:27.743873 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.743854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"]
Jul 16 21:56:27.825311 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.825272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"]
Jul 16 21:56:27.831462 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.831436    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:27.856500 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.856471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-826zp\" (UniqueName: \"kubernetes.io/projected/c817e407-21c5-4917-a80e-e95a83843105-kube-api-access-826zp\") pod \"limitador-limitador-64c94fd64f-qlgtk\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:27.856628 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.856515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c817e407-21c5-4917-a80e-e95a83843105-config-file\") pod \"limitador-limitador-64c94fd64f-qlgtk\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:27.933635 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.933606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:27.957767 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.957738    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-826zp\" (UniqueName: \"kubernetes.io/projected/c817e407-21c5-4917-a80e-e95a83843105-kube-api-access-826zp\") pod \"limitador-limitador-64c94fd64f-qlgtk\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:27.957901 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.957782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c817e407-21c5-4917-a80e-e95a83843105-config-file\") pod \"limitador-limitador-64c94fd64f-qlgtk\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:27.958397 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.958380    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c817e407-21c5-4917-a80e-e95a83843105-config-file\") pod \"limitador-limitador-64c94fd64f-qlgtk\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:27.968431 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:27.968411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-826zp\" (UniqueName: \"kubernetes.io/projected/c817e407-21c5-4917-a80e-e95a83843105-kube-api-access-826zp\") pod \"limitador-limitador-64c94fd64f-qlgtk\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:28.039715 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.039633    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:28.212520 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.212490    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"]
Jul 16 21:56:28.216229 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:56:28.216206    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc817e407_21c5_4917_a80e_e95a83843105.slice/crio-8769f6af4fe88122be1c5f2396a495f385c4581e8fb90fba25090d5704c8dec5 WatchSource:0}: Error finding container 8769f6af4fe88122be1c5f2396a495f385c4581e8fb90fba25090d5704c8dec5: Status 404 returned error can't find the container with id 8769f6af4fe88122be1c5f2396a495f385c4581e8fb90fba25090d5704c8dec5
Jul 16 21:56:28.280159 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.280120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk" event={"ID":"c817e407-21c5-4917-a80e-e95a83843105","Type":"ContainerStarted","Data":"8769f6af4fe88122be1c5f2396a495f385c4581e8fb90fba25090d5704c8dec5"}
Jul 16 21:56:28.669481 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.669439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:28.746222 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.746191    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:28.775119 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.775092    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wx94l"]
Jul 16 21:56:28.779682 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.779658    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wx94l"
Jul 16 21:56:28.782466 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.782438    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tr874\""
Jul 16 21:56:28.787483 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.787460    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wx94l"]
Jul 16 21:56:28.865497 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.865461    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttm9x\" (UniqueName: \"kubernetes.io/projected/70c50b13-4874-4dd1-87d4-3d8d46f73e83-kube-api-access-ttm9x\") pod \"authorino-75d556bd77-wx94l\" (UID: \"70c50b13-4874-4dd1-87d4-3d8d46f73e83\") " pod="kuadrant-system/authorino-75d556bd77-wx94l"
Jul 16 21:56:28.966300 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.966275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttm9x\" (UniqueName: \"kubernetes.io/projected/70c50b13-4874-4dd1-87d4-3d8d46f73e83-kube-api-access-ttm9x\") pod \"authorino-75d556bd77-wx94l\" (UID: \"70c50b13-4874-4dd1-87d4-3d8d46f73e83\") " pod="kuadrant-system/authorino-75d556bd77-wx94l"
Jul 16 21:56:28.975730 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:28.975708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttm9x\" (UniqueName: \"kubernetes.io/projected/70c50b13-4874-4dd1-87d4-3d8d46f73e83-kube-api-access-ttm9x\") pod \"authorino-75d556bd77-wx94l\" (UID: \"70c50b13-4874-4dd1-87d4-3d8d46f73e83\") " pod="kuadrant-system/authorino-75d556bd77-wx94l"
Jul 16 21:56:29.091370 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:29.091332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wx94l"
Jul 16 21:56:29.286446 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:29.286421    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wx94l"]
Jul 16 21:56:29.289666 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:56:29.289638    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70c50b13_4874_4dd1_87d4_3d8d46f73e83.slice/crio-0bc2d4f7ac80d9e57d043f92e4ba3488a14b2c50eca5c46b1960e8842beb3c62 WatchSource:0}: Error finding container 0bc2d4f7ac80d9e57d043f92e4ba3488a14b2c50eca5c46b1960e8842beb3c62: Status 404 returned error can't find the container with id 0bc2d4f7ac80d9e57d043f92e4ba3488a14b2c50eca5c46b1960e8842beb3c62
Jul 16 21:56:29.536530 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:29.536439    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:29.631435 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:29.631398    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:29.924929 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:29.924833    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:29.998153 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:29.998117    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:30.294134 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:30.294096    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wx94l" event={"ID":"70c50b13-4874-4dd1-87d4-3d8d46f73e83","Type":"ContainerStarted","Data":"0bc2d4f7ac80d9e57d043f92e4ba3488a14b2c50eca5c46b1960e8842beb3c62"}
Jul 16 21:56:31.303562 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:31.303524    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk" event={"ID":"c817e407-21c5-4917-a80e-e95a83843105","Type":"ContainerStarted","Data":"74b9ba9f4f3e59fb122dbbc4792b5fa988d83fa815d0a480d4f64a4008abbb21"}
Jul 16 21:56:31.304084 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:31.303832    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:31.323777 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:31.323732    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk" podStartSLOduration=1.339020852 podStartE2EDuration="4.323715793s" totalImagesPullingTime="2.984694941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:56:31.322103557 +0000 UTC m=+457.711377364" watchObservedRunningTime="2026-07-16 21:56:31.323715793 +0000 UTC m=+457.712989618"
Jul 16 21:56:33.012528 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:33.012493    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:33.131425 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:33.131389    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:34.320471 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:34.320434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wx94l" event={"ID":"70c50b13-4874-4dd1-87d4-3d8d46f73e83","Type":"ContainerStarted","Data":"61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2"}
Jul 16 21:56:34.346752 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:34.346707    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wx94l" podStartSLOduration=2.33675017 podStartE2EDuration="6.346691178s" totalImagesPullingTime="4.009941008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:56:34.345292318 +0000 UTC m=+460.734566106" watchObservedRunningTime="2026-07-16 21:56:34.346691178 +0000 UTC m=+460.735965007"
Jul 16 21:56:42.309882 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:42.309853    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:42.900321 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:42.900289    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"]
Jul 16 21:56:42.900502 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:42.900478    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk" podUID="c817e407-21c5-4917-a80e-e95a83843105" containerName="limitador" containerID="cri-o://74b9ba9f4f3e59fb122dbbc4792b5fa988d83fa815d0a480d4f64a4008abbb21" gracePeriod=30
Jul 16 21:56:43.363874 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.363834    2564 generic.go:356] "Generic (PLEG): container finished" podID="c817e407-21c5-4917-a80e-e95a83843105" containerID="74b9ba9f4f3e59fb122dbbc4792b5fa988d83fa815d0a480d4f64a4008abbb21" exitCode=0
Jul 16 21:56:43.364238 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.363901    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk" event={"ID":"c817e407-21c5-4917-a80e-e95a83843105","Type":"ContainerDied","Data":"74b9ba9f4f3e59fb122dbbc4792b5fa988d83fa815d0a480d4f64a4008abbb21"}
Jul 16 21:56:43.452456 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.452432    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:43.594966 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.594880    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c817e407-21c5-4917-a80e-e95a83843105-config-file\") pod \"c817e407-21c5-4917-a80e-e95a83843105\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") "
Jul 16 21:56:43.594966 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.594954    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-826zp\" (UniqueName: \"kubernetes.io/projected/c817e407-21c5-4917-a80e-e95a83843105-kube-api-access-826zp\") pod \"c817e407-21c5-4917-a80e-e95a83843105\" (UID: \"c817e407-21c5-4917-a80e-e95a83843105\") "
Jul 16 21:56:43.595260 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.595238    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c817e407-21c5-4917-a80e-e95a83843105-config-file" (OuterVolumeSpecName: "config-file") pod "c817e407-21c5-4917-a80e-e95a83843105" (UID: "c817e407-21c5-4917-a80e-e95a83843105"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:56:43.597094 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.597072    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c817e407-21c5-4917-a80e-e95a83843105-kube-api-access-826zp" (OuterVolumeSpecName: "kube-api-access-826zp") pod "c817e407-21c5-4917-a80e-e95a83843105" (UID: "c817e407-21c5-4917-a80e-e95a83843105"). InnerVolumeSpecName "kube-api-access-826zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:56:43.696436 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.696407    2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c817e407-21c5-4917-a80e-e95a83843105-config-file\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:56:43.696436 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:43.696430    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-826zp\" (UniqueName: \"kubernetes.io/projected/c817e407-21c5-4917-a80e-e95a83843105-kube-api-access-826zp\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:56:44.370362 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:44.370331    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"
Jul 16 21:56:44.370793 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:44.370330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qlgtk" event={"ID":"c817e407-21c5-4917-a80e-e95a83843105","Type":"ContainerDied","Data":"8769f6af4fe88122be1c5f2396a495f385c4581e8fb90fba25090d5704c8dec5"}
Jul 16 21:56:44.370793 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:44.370466    2564 scope.go:117] "RemoveContainer" containerID="74b9ba9f4f3e59fb122dbbc4792b5fa988d83fa815d0a480d4f64a4008abbb21"
Jul 16 21:56:44.392044 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:44.391978    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"]
Jul 16 21:56:44.402868 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:44.402844    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qlgtk"]
Jul 16 21:56:46.125504 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:46.125467    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c817e407-21c5-4917-a80e-e95a83843105" path="/var/lib/kubelet/pods/c817e407-21c5-4917-a80e-e95a83843105/volumes"
Jul 16 21:56:48.904608 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:48.904556    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:48.998550 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:48.998509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:49.279803 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.279774    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"]
Jul 16 21:56:49.280170 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.280157    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c817e407-21c5-4917-a80e-e95a83843105" containerName="limitador"
Jul 16 21:56:49.280222 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.280172    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c817e407-21c5-4917-a80e-e95a83843105" containerName="limitador"
Jul 16 21:56:49.280256 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.280228    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c817e407-21c5-4917-a80e-e95a83843105" containerName="limitador"
Jul 16 21:56:49.283346 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.283330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.288059 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.288035    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 21:56:49.288170 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.288107    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:56:49.288885 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.288859    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-mz844\""
Jul 16 21:56:49.288885 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.288880    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 16 21:56:49.301541 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.301517    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"]
Jul 16 21:56:49.310266 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.310237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:56:49.345357 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.345328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d04ae75-9d74-4e95-a1f6-2d24a606d90c-data\") pod \"postgres-69fb4594cf-27jcj\" (UID: \"5d04ae75-9d74-4e95-a1f6-2d24a606d90c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.345357 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.345358    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2pg6\" (UniqueName: \"kubernetes.io/projected/5d04ae75-9d74-4e95-a1f6-2d24a606d90c-kube-api-access-s2pg6\") pod \"postgres-69fb4594cf-27jcj\" (UID: \"5d04ae75-9d74-4e95-a1f6-2d24a606d90c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.445840 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.445811    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d04ae75-9d74-4e95-a1f6-2d24a606d90c-data\") pod \"postgres-69fb4594cf-27jcj\" (UID: \"5d04ae75-9d74-4e95-a1f6-2d24a606d90c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.445840 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.445838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2pg6\" (UniqueName: \"kubernetes.io/projected/5d04ae75-9d74-4e95-a1f6-2d24a606d90c-kube-api-access-s2pg6\") pod \"postgres-69fb4594cf-27jcj\" (UID: \"5d04ae75-9d74-4e95-a1f6-2d24a606d90c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.446201 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.446181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d04ae75-9d74-4e95-a1f6-2d24a606d90c-data\") pod \"postgres-69fb4594cf-27jcj\" (UID: \"5d04ae75-9d74-4e95-a1f6-2d24a606d90c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.455816 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.455787    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2pg6\" (UniqueName: \"kubernetes.io/projected/5d04ae75-9d74-4e95-a1f6-2d24a606d90c-kube-api-access-s2pg6\") pod \"postgres-69fb4594cf-27jcj\" (UID: \"5d04ae75-9d74-4e95-a1f6-2d24a606d90c\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.593265 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.593191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:49.733994 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:49.733959    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"]
Jul 16 21:56:49.737821 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:56:49.737794    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d04ae75_9d74_4e95_a1f6_2d24a606d90c.slice/crio-8a631e0be5a85c1931ff8170863df8bbe75410025e24ad0541d2eb54ecf25cbe WatchSource:0}: Error finding container 8a631e0be5a85c1931ff8170863df8bbe75410025e24ad0541d2eb54ecf25cbe: Status 404 returned error can't find the container with id 8a631e0be5a85c1931ff8170863df8bbe75410025e24ad0541d2eb54ecf25cbe
Jul 16 21:56:50.399931 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:50.399877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj" event={"ID":"5d04ae75-9d74-4e95-a1f6-2d24a606d90c","Type":"ContainerStarted","Data":"8a631e0be5a85c1931ff8170863df8bbe75410025e24ad0541d2eb54ecf25cbe"}
Jul 16 21:56:54.648239 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:54.648215    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 16 21:56:55.432856 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:55.432815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj" event={"ID":"5d04ae75-9d74-4e95-a1f6-2d24a606d90c","Type":"ContainerStarted","Data":"b124ca6fa64e07f67b0142819e353f27386a0f362ed9cafbfb406fc070b209c2"}
Jul 16 21:56:55.433066 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:55.433027    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:56:55.456709 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:56:55.456653    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj" podStartSLOduration=1.551362423 podStartE2EDuration="6.456635672s" totalImagesPullingTime="4.905273249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:56:55.452127657 +0000 UTC m=+481.841401442" watchObservedRunningTime="2026-07-16 21:56:55.456635672 +0000 UTC m=+481.845909459"
Jul 16 21:57:01.467576 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:01.467541    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-27jcj"
Jul 16 21:57:01.855019 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:01.854930    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:02.007891 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:02.007861    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:04.441986 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:04.441951    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wx94l"]
Jul 16 21:57:04.442488 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:04.442139    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wx94l" podUID="70c50b13-4874-4dd1-87d4-3d8d46f73e83" containerName="authorino" containerID="cri-o://61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2" gracePeriod=30
Jul 16 21:57:04.688136 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:04.688109    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wx94l"
Jul 16 21:57:04.782984 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:04.782872    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttm9x\" (UniqueName: \"kubernetes.io/projected/70c50b13-4874-4dd1-87d4-3d8d46f73e83-kube-api-access-ttm9x\") pod \"70c50b13-4874-4dd1-87d4-3d8d46f73e83\" (UID: \"70c50b13-4874-4dd1-87d4-3d8d46f73e83\") "
Jul 16 21:57:04.785117 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:04.785084    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c50b13-4874-4dd1-87d4-3d8d46f73e83-kube-api-access-ttm9x" (OuterVolumeSpecName: "kube-api-access-ttm9x") pod "70c50b13-4874-4dd1-87d4-3d8d46f73e83" (UID: "70c50b13-4874-4dd1-87d4-3d8d46f73e83"). InnerVolumeSpecName "kube-api-access-ttm9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:57:04.883540 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:04.883508    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttm9x\" (UniqueName: \"kubernetes.io/projected/70c50b13-4874-4dd1-87d4-3d8d46f73e83-kube-api-access-ttm9x\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:57:05.484065 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.484030    2564 generic.go:356] "Generic (PLEG): container finished" podID="70c50b13-4874-4dd1-87d4-3d8d46f73e83" containerID="61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2" exitCode=0
Jul 16 21:57:05.484523 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.484083    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wx94l"
Jul 16 21:57:05.484523 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.484115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wx94l" event={"ID":"70c50b13-4874-4dd1-87d4-3d8d46f73e83","Type":"ContainerDied","Data":"61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2"}
Jul 16 21:57:05.484523 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.484159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wx94l" event={"ID":"70c50b13-4874-4dd1-87d4-3d8d46f73e83","Type":"ContainerDied","Data":"0bc2d4f7ac80d9e57d043f92e4ba3488a14b2c50eca5c46b1960e8842beb3c62"}
Jul 16 21:57:05.484523 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.484175    2564 scope.go:117] "RemoveContainer" containerID="61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2"
Jul 16 21:57:05.493567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.493548    2564 scope.go:117] "RemoveContainer" containerID="61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2"
Jul 16 21:57:05.493840 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:57:05.493820    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2\": container with ID starting with 61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2 not found: ID does not exist" containerID="61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2"
Jul 16 21:57:05.493893 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.493848    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2"} err="failed to get container status \"61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2\": rpc error: code = NotFound desc = could not find container \"61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2\": container with ID starting with 61b038cc2bb457f8ab7b887a0f41f8a295371c0664c59a0b1a7068be91f2dae2 not found: ID does not exist"
Jul 16 21:57:05.506396 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.506373    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wx94l"]
Jul 16 21:57:05.511142 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:05.511122    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wx94l"]
Jul 16 21:57:06.124767 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:06.124735    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c50b13-4874-4dd1-87d4-3d8d46f73e83" path="/var/lib/kubelet/pods/70c50b13-4874-4dd1-87d4-3d8d46f73e83/volumes"
Jul 16 21:57:09.529572 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.529539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:09.544864 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.544834    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c845df9c8-vq8ln"]
Jul 16 21:57:09.545209 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.545196    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c50b13-4874-4dd1-87d4-3d8d46f73e83" containerName="authorino"
Jul 16 21:57:09.545254 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.545211    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c50b13-4874-4dd1-87d4-3d8d46f73e83" containerName="authorino"
Jul 16 21:57:09.545314 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.545275    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c50b13-4874-4dd1-87d4-3d8d46f73e83" containerName="authorino"
Jul 16 21:57:09.547814 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.547796    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:09.550606 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.550582    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 21:57:09.550923 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.550899    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d7qf9\""
Jul 16 21:57:09.551532 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.551519    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 21:57:09.565476 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.565451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c845df9c8-vq8ln"]
Jul 16 21:57:09.618142 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.618110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9525\" (UniqueName: \"kubernetes.io/projected/945d097a-d038-44ed-9244-95bd1ad1dfff-kube-api-access-c9525\") pod \"maas-controller-6c845df9c8-vq8ln\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") " pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:09.618282 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.618169    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert\") pod \"maas-controller-6c845df9c8-vq8ln\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") " pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:09.655655 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.655623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:09.718889 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.718859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9525\" (UniqueName: \"kubernetes.io/projected/945d097a-d038-44ed-9244-95bd1ad1dfff-kube-api-access-c9525\") pod \"maas-controller-6c845df9c8-vq8ln\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") " pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:09.719049 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.718949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert\") pod \"maas-controller-6c845df9c8-vq8ln\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") " pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:09.719093 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:57:09.719056    2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 16 21:57:09.719138 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:57:09.719127    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert podName:945d097a-d038-44ed-9244-95bd1ad1dfff nodeName:}" failed. No retries permitted until 2026-07-16 21:57:10.219112268 +0000 UTC m=+496.608386033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert") pod "maas-controller-6c845df9c8-vq8ln" (UID: "945d097a-d038-44ed-9244-95bd1ad1dfff") : secret "maas-controller-webhook-cert" not found
Jul 16 21:57:09.728478 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.728446    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9525\" (UniqueName: \"kubernetes.io/projected/945d097a-d038-44ed-9244-95bd1ad1dfff-kube-api-access-c9525\") pod \"maas-controller-6c845df9c8-vq8ln\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") " pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:09.760950 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.760895    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:09.772682 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.772657    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-mx4vs"]
Jul 16 21:57:09.775141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.775121    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:09.787266 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.787244    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-mx4vs"]
Jul 16 21:57:09.819686 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.819660    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgl6w\" (UniqueName: \"kubernetes.io/projected/4231a980-36df-4b94-a964-2e5c29048243-kube-api-access-kgl6w\") pod \"maas-controller-6f4d5cdddc-mx4vs\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") " pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:09.819787 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.819709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4231a980-36df-4b94-a964-2e5c29048243-webhook-cert\") pod \"maas-controller-6f4d5cdddc-mx4vs\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") " pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:09.868690 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.868666    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:09.921018 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.920988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgl6w\" (UniqueName: \"kubernetes.io/projected/4231a980-36df-4b94-a964-2e5c29048243-kube-api-access-kgl6w\") pod \"maas-controller-6f4d5cdddc-mx4vs\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") " pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:09.921121 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.921045    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4231a980-36df-4b94-a964-2e5c29048243-webhook-cert\") pod \"maas-controller-6f4d5cdddc-mx4vs\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") " pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:09.923570 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.923544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4231a980-36df-4b94-a964-2e5c29048243-webhook-cert\") pod \"maas-controller-6f4d5cdddc-mx4vs\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") " pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:09.932845 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:09.932824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgl6w\" (UniqueName: \"kubernetes.io/projected/4231a980-36df-4b94-a964-2e5c29048243-kube-api-access-kgl6w\") pod \"maas-controller-6f4d5cdddc-mx4vs\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") " pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:10.085232 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:10.085158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:10.223385 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:10.223354    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-mx4vs"]
Jul 16 21:57:10.223796 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:10.223769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert\") pod \"maas-controller-6c845df9c8-vq8ln\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") " pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:10.226489 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:57:10.226467    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4231a980_36df_4b94_a964_2e5c29048243.slice/crio-6d44e3fee24f295ff7148582830b3391bace707a66772797a596365ab2bbcd95 WatchSource:0}: Error finding container 6d44e3fee24f295ff7148582830b3391bace707a66772797a596365ab2bbcd95: Status 404 returned error can't find the container with id 6d44e3fee24f295ff7148582830b3391bace707a66772797a596365ab2bbcd95
Jul 16 21:57:10.226578 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:10.226540    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert\") pod \"maas-controller-6c845df9c8-vq8ln\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") " pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:10.457561 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:10.457527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:10.511778 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:10.511746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs" event={"ID":"4231a980-36df-4b94-a964-2e5c29048243","Type":"ContainerStarted","Data":"6d44e3fee24f295ff7148582830b3391bace707a66772797a596365ab2bbcd95"}
Jul 16 21:57:10.592812 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:10.592786    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c845df9c8-vq8ln"]
Jul 16 21:57:10.594871 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:57:10.594849    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945d097a_d038_44ed_9244_95bd1ad1dfff.slice/crio-2640b81dcb7026177ac199b797f2c5115ec80c92358d42a81f350f4830254208 WatchSource:0}: Error finding container 2640b81dcb7026177ac199b797f2c5115ec80c92358d42a81f350f4830254208: Status 404 returned error can't find the container with id 2640b81dcb7026177ac199b797f2c5115ec80c92358d42a81f350f4830254208
Jul 16 21:57:11.526457 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:11.526382    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c845df9c8-vq8ln" event={"ID":"945d097a-d038-44ed-9244-95bd1ad1dfff","Type":"ContainerStarted","Data":"2640b81dcb7026177ac199b797f2c5115ec80c92358d42a81f350f4830254208"}
Jul 16 21:57:13.153133 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.153100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:13.248665 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.248632    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:13.537682 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.537589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c845df9c8-vq8ln" event={"ID":"945d097a-d038-44ed-9244-95bd1ad1dfff","Type":"ContainerStarted","Data":"d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87"}
Jul 16 21:57:13.537682 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.537679    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:13.539099 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.539062    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs" event={"ID":"4231a980-36df-4b94-a964-2e5c29048243","Type":"ContainerStarted","Data":"cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7"}
Jul 16 21:57:13.539240 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.539190    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:13.579058 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.579014    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c845df9c8-vq8ln" podStartSLOduration=2.226447461 podStartE2EDuration="4.579001347s" totalImagesPullingTime="2.352553886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:13.577778095 +0000 UTC m=+499.967051881" watchObservedRunningTime="2026-07-16 21:57:13.579001347 +0000 UTC m=+499.968275133"
Jul 16 21:57:13.610489 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:13.610449    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs" podStartSLOduration=1.8995972559999998 podStartE2EDuration="4.610435565s" totalImagesPullingTime="2.710838309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:13.60888082 +0000 UTC m=+499.998154605" watchObservedRunningTime="2026-07-16 21:57:13.610435565 +0000 UTC m=+499.999709385"
Jul 16 21:57:24.545711 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.545683    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:24.546190 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.545752    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 21:57:24.622794 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.619599    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c845df9c8-vq8ln"]
Jul 16 21:57:24.622794 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.622144    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c845df9c8-vq8ln" podUID="945d097a-d038-44ed-9244-95bd1ad1dfff" containerName="manager" containerID="cri-o://d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87" gracePeriod=30
Jul 16 21:57:24.878496 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.878473    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:24.948567 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.948540    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert\") pod \"945d097a-d038-44ed-9244-95bd1ad1dfff\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") "
Jul 16 21:57:24.948737 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.948659    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9525\" (UniqueName: \"kubernetes.io/projected/945d097a-d038-44ed-9244-95bd1ad1dfff-kube-api-access-c9525\") pod \"945d097a-d038-44ed-9244-95bd1ad1dfff\" (UID: \"945d097a-d038-44ed-9244-95bd1ad1dfff\") "
Jul 16 21:57:24.951045 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.951012    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "945d097a-d038-44ed-9244-95bd1ad1dfff" (UID: "945d097a-d038-44ed-9244-95bd1ad1dfff"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:57:24.951171 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:24.951074    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945d097a-d038-44ed-9244-95bd1ad1dfff-kube-api-access-c9525" (OuterVolumeSpecName: "kube-api-access-c9525") pod "945d097a-d038-44ed-9244-95bd1ad1dfff" (UID: "945d097a-d038-44ed-9244-95bd1ad1dfff"). InnerVolumeSpecName "kube-api-access-c9525". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:57:25.049878 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.049844    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/945d097a-d038-44ed-9244-95bd1ad1dfff-webhook-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:57:25.049878 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.049873    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9525\" (UniqueName: \"kubernetes.io/projected/945d097a-d038-44ed-9244-95bd1ad1dfff-kube-api-access-c9525\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 21:57:25.400089 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.400050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:25.597038 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.597007    2564 generic.go:356] "Generic (PLEG): container finished" podID="945d097a-d038-44ed-9244-95bd1ad1dfff" containerID="d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87" exitCode=0
Jul 16 21:57:25.597434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.597054    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c845df9c8-vq8ln"
Jul 16 21:57:25.597434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.597103    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c845df9c8-vq8ln" event={"ID":"945d097a-d038-44ed-9244-95bd1ad1dfff","Type":"ContainerDied","Data":"d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87"}
Jul 16 21:57:25.597434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.597155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c845df9c8-vq8ln" event={"ID":"945d097a-d038-44ed-9244-95bd1ad1dfff","Type":"ContainerDied","Data":"2640b81dcb7026177ac199b797f2c5115ec80c92358d42a81f350f4830254208"}
Jul 16 21:57:25.597434 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.597179    2564 scope.go:117] "RemoveContainer" containerID="d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87"
Jul 16 21:57:25.606099 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.606082    2564 scope.go:117] "RemoveContainer" containerID="d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87"
Jul 16 21:57:25.606355 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:57:25.606336    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87\": container with ID starting with d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87 not found: ID does not exist" containerID="d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87"
Jul 16 21:57:25.606562 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.606364    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87"} err="failed to get container status \"d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87\": rpc error: code = NotFound desc = could not find container \"d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87\": container with ID starting with d3079defeb4f96c7f4b5be527128d80ac4aaedd9d28b902f44ba1f623944df87 not found: ID does not exist"
Jul 16 21:57:25.620943 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.620897    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c845df9c8-vq8ln"]
Jul 16 21:57:25.626050 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:25.626024    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c845df9c8-vq8ln"]
Jul 16 21:57:26.125174 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:26.125143    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945d097a-d038-44ed-9244-95bd1ad1dfff" path="/var/lib/kubelet/pods/945d097a-d038-44ed-9244-95bd1ad1dfff/volumes"
Jul 16 21:57:37.570653 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:37.570610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:44.485158 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:44.485096    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:44.583069 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:44.582963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:45.121327 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.121292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:45.258569 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.258535    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"]
Jul 16 21:57:45.258871 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.258859    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945d097a-d038-44ed-9244-95bd1ad1dfff" containerName="manager"
Jul 16 21:57:45.258930 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.258872    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="945d097a-d038-44ed-9244-95bd1ad1dfff" containerName="manager"
Jul 16 21:57:45.258973 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.258964    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="945d097a-d038-44ed-9244-95bd1ad1dfff" containerName="manager"
Jul 16 21:57:45.282147 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.282118    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"]
Jul 16 21:57:45.282300 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.282226    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.284992 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.284959    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 16 21:57:45.285385 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.285362    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-td7dh\""
Jul 16 21:57:45.420353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.420278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94941e6c-ed92-4e91-876d-f2db63e8d1b2-openshift-service-ca\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.420353 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.420332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qssz\" (UniqueName: \"kubernetes.io/projected/94941e6c-ed92-4e91-876d-f2db63e8d1b2-kube-api-access-5qssz\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.420535 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.420410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94941e6c-ed92-4e91-876d-f2db63e8d1b2-maas-api-tls\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.521720 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.521687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qssz\" (UniqueName: \"kubernetes.io/projected/94941e6c-ed92-4e91-876d-f2db63e8d1b2-kube-api-access-5qssz\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.522124 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.521730    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94941e6c-ed92-4e91-876d-f2db63e8d1b2-maas-api-tls\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.522124 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.521779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94941e6c-ed92-4e91-876d-f2db63e8d1b2-openshift-service-ca\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.522124 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:57:45.521925    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 16 21:57:45.522124 ip-10-0-128-173 kubenswrapper[2564]: E0716 21:57:45.522000    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94941e6c-ed92-4e91-876d-f2db63e8d1b2-maas-api-tls podName:94941e6c-ed92-4e91-876d-f2db63e8d1b2 nodeName:}" failed. No retries permitted until 2026-07-16 21:57:46.021983664 +0000 UTC m=+532.411257428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/94941e6c-ed92-4e91-876d-f2db63e8d1b2-maas-api-tls") pod "maas-api-54d578c6bd-24f5x" (UID: "94941e6c-ed92-4e91-876d-f2db63e8d1b2") : secret "maas-api-serving-cert" not found
Jul 16 21:57:45.522481 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.522465    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94941e6c-ed92-4e91-876d-f2db63e8d1b2-openshift-service-ca\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.535687 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.535662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qssz\" (UniqueName: \"kubernetes.io/projected/94941e6c-ed92-4e91-876d-f2db63e8d1b2-kube-api-access-5qssz\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:45.675563 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.675471    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:45.878483 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.878449    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-d74pp"]
Jul 16 21:57:45.881658 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.881640    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:45.885653 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.885632    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m86rz\""
Jul 16 21:57:45.885757 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.885634    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 16 21:57:45.900511 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.900486    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-d74pp"]
Jul 16 21:57:45.913642 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.913618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:45.986096 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.986063    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nxppb"]
Jul 16 21:57:45.989315 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:45.989294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.003723 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.003698    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nxppb"]
Jul 16 21:57:46.026963 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.026940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94941e6c-ed92-4e91-876d-f2db63e8d1b2-maas-api-tls\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:46.027065 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.026985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/019c2a8a-65a4-44e7-8b8f-05affe600636-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-d74pp\" (UID: \"019c2a8a-65a4-44e7-8b8f-05affe600636\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:46.027065 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.027019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpr58\" (UniqueName: \"kubernetes.io/projected/019c2a8a-65a4-44e7-8b8f-05affe600636-kube-api-access-cpr58\") pod \"payload-pre-processing-7677b774bb-d74pp\" (UID: \"019c2a8a-65a4-44e7-8b8f-05affe600636\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:46.029485 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.029461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94941e6c-ed92-4e91-876d-f2db63e8d1b2-maas-api-tls\") pod \"maas-api-54d578c6bd-24f5x\" (UID: \"94941e6c-ed92-4e91-876d-f2db63e8d1b2\") " pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:46.039428 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.039401    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:46.127947 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.127900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/019c2a8a-65a4-44e7-8b8f-05affe600636-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-d74pp\" (UID: \"019c2a8a-65a4-44e7-8b8f-05affe600636\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:46.128135 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.127958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05730b62-84a5-4146-8ff3-4a274b9bd942-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nxppb\" (UID: \"05730b62-84a5-4146-8ff3-4a274b9bd942\") " pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.128135 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.127989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpr58\" (UniqueName: \"kubernetes.io/projected/019c2a8a-65a4-44e7-8b8f-05affe600636-kube-api-access-cpr58\") pod \"payload-pre-processing-7677b774bb-d74pp\" (UID: \"019c2a8a-65a4-44e7-8b8f-05affe600636\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:46.128135 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.128065    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7th8\" (UniqueName: \"kubernetes.io/projected/05730b62-84a5-4146-8ff3-4a274b9bd942-kube-api-access-q7th8\") pod \"payload-processing-5bf47cd988-nxppb\" (UID: \"05730b62-84a5-4146-8ff3-4a274b9bd942\") " pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.128586 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.128554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/019c2a8a-65a4-44e7-8b8f-05affe600636-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-d74pp\" (UID: \"019c2a8a-65a4-44e7-8b8f-05affe600636\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:46.149613 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.149585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpr58\" (UniqueName: \"kubernetes.io/projected/019c2a8a-65a4-44e7-8b8f-05affe600636-kube-api-access-cpr58\") pod \"payload-pre-processing-7677b774bb-d74pp\" (UID: \"019c2a8a-65a4-44e7-8b8f-05affe600636\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:46.192525 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.192486    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:46.193581 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.193561    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:46.228853 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.228820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7th8\" (UniqueName: \"kubernetes.io/projected/05730b62-84a5-4146-8ff3-4a274b9bd942-kube-api-access-q7th8\") pod \"payload-processing-5bf47cd988-nxppb\" (UID: \"05730b62-84a5-4146-8ff3-4a274b9bd942\") " pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.229056 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.228891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05730b62-84a5-4146-8ff3-4a274b9bd942-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nxppb\" (UID: \"05730b62-84a5-4146-8ff3-4a274b9bd942\") " pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.230128 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.230080    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05730b62-84a5-4146-8ff3-4a274b9bd942-plugins-config-volume\") pod \"payload-processing-5bf47cd988-nxppb\" (UID: \"05730b62-84a5-4146-8ff3-4a274b9bd942\") " pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.244458 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.244363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7th8\" (UniqueName: \"kubernetes.io/projected/05730b62-84a5-4146-8ff3-4a274b9bd942-kube-api-access-q7th8\") pod \"payload-processing-5bf47cd988-nxppb\" (UID: \"05730b62-84a5-4146-8ff3-4a274b9bd942\") " pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.299670 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.299590    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:46.394754 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.394716    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"]
Jul 16 21:57:46.451803 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.451778    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-nxppb"]
Jul 16 21:57:46.454967 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:57:46.454905    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05730b62_84a5_4146_8ff3_4a274b9bd942.slice/crio-a301e2c4ebbd1ea44e4fdb27f72a639bfbbd75dfc32db69214c42911c7faddb0 WatchSource:0}: Error finding container a301e2c4ebbd1ea44e4fdb27f72a639bfbbd75dfc32db69214c42911c7faddb0: Status 404 returned error can't find the container with id a301e2c4ebbd1ea44e4fdb27f72a639bfbbd75dfc32db69214c42911c7faddb0
Jul 16 21:57:46.555097 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.555070    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-d74pp"]
Jul 16 21:57:46.557819 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:57:46.557794    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019c2a8a_65a4_44e7_8b8f_05affe600636.slice/crio-8eb58474dc68f3f94f351258cbc5ad3520de2fde6ade7496afc5a5e28e86e9ce WatchSource:0}: Error finding container 8eb58474dc68f3f94f351258cbc5ad3520de2fde6ade7496afc5a5e28e86e9ce: Status 404 returned error can't find the container with id 8eb58474dc68f3f94f351258cbc5ad3520de2fde6ade7496afc5a5e28e86e9ce
Jul 16 21:57:46.697920 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.697887    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp" event={"ID":"019c2a8a-65a4-44e7-8b8f-05affe600636","Type":"ContainerStarted","Data":"8eb58474dc68f3f94f351258cbc5ad3520de2fde6ade7496afc5a5e28e86e9ce"}
Jul 16 21:57:46.699259 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.699234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-nxppb" event={"ID":"05730b62-84a5-4146-8ff3-4a274b9bd942","Type":"ContainerStarted","Data":"a301e2c4ebbd1ea44e4fdb27f72a639bfbbd75dfc32db69214c42911c7faddb0"}
Jul 16 21:57:46.700492 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:46.700467    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x" event={"ID":"94941e6c-ed92-4e91-876d-f2db63e8d1b2","Type":"ContainerStarted","Data":"6193fd117668c3d21813a1723ca2ae0c7e3effb337a2c82e8604b73ad6fdc30d"}
Jul 16 21:57:49.725894 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.725623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp" event={"ID":"019c2a8a-65a4-44e7-8b8f-05affe600636","Type":"ContainerStarted","Data":"fb47789481c1fa7230f0b9b0279587bddb37a70df26db9e73e88b69241b2085d"}
Jul 16 21:57:49.725894 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.725747    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:57:49.727716 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.727681    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-nxppb" event={"ID":"05730b62-84a5-4146-8ff3-4a274b9bd942","Type":"ContainerStarted","Data":"353f8ad94b20163e87658ed99751cd86770aa0c692e53a7e8c15e04fa32dae31"}
Jul 16 21:57:49.727982 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.727948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:57:49.729395 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.729372    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x" event={"ID":"94941e6c-ed92-4e91-876d-f2db63e8d1b2","Type":"ContainerStarted","Data":"0a3050283bb620f8408791446baee0d479590500d8ae62985fc2f268d7200086"}
Jul 16 21:57:49.729636 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.729617    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:49.745658 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.745596    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp" podStartSLOduration=2.042513289 podStartE2EDuration="4.745578961s" totalImagesPullingTime="2.703065672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:49.742423623 +0000 UTC m=+536.131697432" watchObservedRunningTime="2026-07-16 21:57:49.745578961 +0000 UTC m=+536.134852749"
Jul 16 21:57:49.759979 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.759933    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-nxppb" podStartSLOduration=1.9472985870000001 podStartE2EDuration="4.759906337s" totalImagesPullingTime="2.81260775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:49.759451722 +0000 UTC m=+536.148725532" watchObservedRunningTime="2026-07-16 21:57:49.759906337 +0000 UTC m=+536.149180122"
Jul 16 21:57:49.781001 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:49.780952    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x" podStartSLOduration=1.9810015970000001 podStartE2EDuration="4.780939443s" totalImagesPullingTime="2.799937846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:49.779884674 +0000 UTC m=+536.169158460" watchObservedRunningTime="2026-07-16 21:57:49.780939443 +0000 UTC m=+536.170213229"
Jul 16 21:57:55.739986 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:55.739958    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54d578c6bd-24f5x"
Jul 16 21:57:56.873223 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:56.873193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:57:57.037373 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:57:57.037332    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:58:00.736722 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:00.736694    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-d74pp"
Jul 16 21:58:00.737193 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:00.736767    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-nxppb"
Jul 16 21:58:07.811513 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.811481    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"]
Jul 16 21:58:07.816436 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.816415    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:07.819204 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.819185    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 21:58:07.820207 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.820190    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-khxnq\""
Jul 16 21:58:07.820275 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.820221    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 16 21:58:07.820275 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.820236    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 21:58:07.825569 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.825548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"]
Jul 16 21:58:07.837339 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.837312    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:58:07.905973 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.905944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsp7t\" (UniqueName: \"kubernetes.io/projected/0a756350-8d5c-4d92-bfa6-086e21b016a4-kube-api-access-lsp7t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:07.906141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.905982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a756350-8d5c-4d92-bfa6-086e21b016a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:07.906141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.906015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:07.906141 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.906108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:07.906357 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.906156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:07.906357 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:07.906216    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007291 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007458 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsp7t\" (UniqueName: \"kubernetes.io/projected/0a756350-8d5c-4d92-bfa6-086e21b016a4-kube-api-access-lsp7t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007458 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a756350-8d5c-4d92-bfa6-086e21b016a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007458 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007633 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007633 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007742 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007742 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007721    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.007823 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.007784    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.009841 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.009814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a756350-8d5c-4d92-bfa6-086e21b016a4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.010127 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.010108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a756350-8d5c-4d92-bfa6-086e21b016a4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.015687 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.015662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsp7t\" (UniqueName: \"kubernetes.io/projected/0a756350-8d5c-4d92-bfa6-086e21b016a4-kube-api-access-lsp7t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx\" (UID: \"0a756350-8d5c-4d92-bfa6-086e21b016a4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.129365 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.129278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:08.269250 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.269176    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"]
Jul 16 21:58:08.271814 ip-10-0-128-173 kubenswrapper[2564]: W0716 21:58:08.271786    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a756350_8d5c_4d92_bfa6_086e21b016a4.slice/crio-f97cb701c0f79577c30d9788f875f4012e3e70502db23d977b1f124c54c368c0 WatchSource:0}: Error finding container f97cb701c0f79577c30d9788f875f4012e3e70502db23d977b1f124c54c368c0: Status 404 returned error can't find the container with id f97cb701c0f79577c30d9788f875f4012e3e70502db23d977b1f124c54c368c0
Jul 16 21:58:08.830748 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:08.830695    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx" event={"ID":"0a756350-8d5c-4d92-bfa6-086e21b016a4","Type":"ContainerStarted","Data":"f97cb701c0f79577c30d9788f875f4012e3e70502db23d977b1f124c54c368c0"}
Jul 16 21:58:09.838093 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:09.838054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx" event={"ID":"0a756350-8d5c-4d92-bfa6-086e21b016a4","Type":"ContainerStarted","Data":"5660c1a30a3731eea6e98bb5d3a2e0d7acbfae9975ba2e077ce399225cc9b1e2"}
Jul 16 21:58:09.858447 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:09.858396    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx" podStartSLOduration=1.803438342 podStartE2EDuration="2.858380255s" totalImagesPullingTime="1.054941913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:58:09.856597401 +0000 UTC m=+556.245871248" watchObservedRunningTime="2026-07-16 21:58:09.858380255 +0000 UTC m=+556.247654081"
Jul 16 21:58:17.499366 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:17.499327    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:58:18.129355 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:18.129322    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:18.129548 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:18.129375    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:18.142342 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:18.142318    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:18.891755 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:18.891729    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx"
Jul 16 21:58:24.602176 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:24.602131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:58:36.517659 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:36.517623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:58:53.892190 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:53.892148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:58:54.043120 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:54.043091    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 21:58:54.043287 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:54.043244    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 21:58:54.048678 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:54.048655    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:58:54.048982 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:54.048962    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 21:58:59.564309 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:59.564267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:58:59.665481 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:58:59.665452    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:59:08.489455 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:59:08.489418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:59:36.588560 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:59:36.588526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:59:36.646131 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:59:36.646103    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:59:55.843879 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:59:55.843822    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 21:59:55.897320 ip-10-0-128-173 kubenswrapper[2564]: I0716 21:59:55.897287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:00:00.171146 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.171115    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"]
Jul 16 22:00:00.173550 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.173533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"
Jul 16 22:00:00.182929 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.177601    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:00:00.190009 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.189982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"
Jul 16 22:00:00.202333 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.202307    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"]
Jul 16 22:00:00.356653 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.356628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"]
Jul 16 22:00:00.358516 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:00:00.358489    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09aade2_0dc6_4d4c_ad80_0f531e91b81f.slice/crio-0d3a8221fcffdfe668d25ad7b64427d8ff8ad7547ebb2bf615a9df17129e3819 WatchSource:0}: Error finding container 0d3a8221fcffdfe668d25ad7b64427d8ff8ad7547ebb2bf615a9df17129e3819: Status 404 returned error can't find the container with id 0d3a8221fcffdfe668d25ad7b64427d8ff8ad7547ebb2bf615a9df17129e3819
Jul 16 22:00:00.360362 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.360344    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:00:00.394717 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:00.394688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt" event={"ID":"a09aade2-0dc6-4d4c-ad80-0f531e91b81f","Type":"ContainerStarted","Data":"0d3a8221fcffdfe668d25ad7b64427d8ff8ad7547ebb2bf615a9df17129e3819"}
Jul 16 22:00:05.421393 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:05.421357    2564 generic.go:356] "Generic (PLEG): container finished" podID="a09aade2-0dc6-4d4c-ad80-0f531e91b81f" containerID="bf005e76034956d2ad9d6f488c24b19766fc4e15ebccbc7ef282b142dde10d75" exitCode=0
Jul 16 22:00:05.421785 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:05.421410    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt" event={"ID":"a09aade2-0dc6-4d4c-ad80-0f531e91b81f","Type":"ContainerDied","Data":"bf005e76034956d2ad9d6f488c24b19766fc4e15ebccbc7ef282b142dde10d75"}
Jul 16 22:00:06.563322 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:06.563296    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"
Jul 16 22:00:07.267397 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.267365    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9bdf84c6c-t7md7"]
Jul 16 22:00:07.267792 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.267778    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a09aade2-0dc6-4d4c-ad80-0f531e91b81f" containerName="cleanup"
Jul 16 22:00:07.267858 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.267795    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a09aade2-0dc6-4d4c-ad80-0f531e91b81f" containerName="cleanup"
Jul 16 22:00:07.267858 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.267857    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a09aade2-0dc6-4d4c-ad80-0f531e91b81f" containerName="cleanup"
Jul 16 22:00:07.270169 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.270148    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.273575 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.273554    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 16 22:00:07.273667 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.273585    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tr874\""
Jul 16 22:00:07.278246 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.278222    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9bdf84c6c-t7md7"]
Jul 16 22:00:07.297760 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.297731    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5fdv\" (UniqueName: \"kubernetes.io/projected/0cc69be4-dc26-4fad-a7c4-1b46ecb29369-kube-api-access-m5fdv\") pod \"authorino-9bdf84c6c-t7md7\" (UID: \"0cc69be4-dc26-4fad-a7c4-1b46ecb29369\") " pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.297904 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.297814    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0cc69be4-dc26-4fad-a7c4-1b46ecb29369-tls-cert\") pod \"authorino-9bdf84c6c-t7md7\" (UID: \"0cc69be4-dc26-4fad-a7c4-1b46ecb29369\") " pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.399053 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.399023    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0cc69be4-dc26-4fad-a7c4-1b46ecb29369-tls-cert\") pod \"authorino-9bdf84c6c-t7md7\" (UID: \"0cc69be4-dc26-4fad-a7c4-1b46ecb29369\") " pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.399213 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.399064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5fdv\" (UniqueName: \"kubernetes.io/projected/0cc69be4-dc26-4fad-a7c4-1b46ecb29369-kube-api-access-m5fdv\") pod \"authorino-9bdf84c6c-t7md7\" (UID: \"0cc69be4-dc26-4fad-a7c4-1b46ecb29369\") " pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.401766 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.401735    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0cc69be4-dc26-4fad-a7c4-1b46ecb29369-tls-cert\") pod \"authorino-9bdf84c6c-t7md7\" (UID: \"0cc69be4-dc26-4fad-a7c4-1b46ecb29369\") " pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.407530 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.407509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5fdv\" (UniqueName: \"kubernetes.io/projected/0cc69be4-dc26-4fad-a7c4-1b46ecb29369-kube-api-access-m5fdv\") pod \"authorino-9bdf84c6c-t7md7\" (UID: \"0cc69be4-dc26-4fad-a7c4-1b46ecb29369\") " pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.432061 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.432033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt" event={"ID":"a09aade2-0dc6-4d4c-ad80-0f531e91b81f","Type":"ContainerDied","Data":"0d3a8221fcffdfe668d25ad7b64427d8ff8ad7547ebb2bf615a9df17129e3819"}
Jul 16 22:00:07.432061 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.432052    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"
Jul 16 22:00:07.432212 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.432080    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d3a8221fcffdfe668d25ad7b64427d8ff8ad7547ebb2bf615a9df17129e3819"
Jul 16 22:00:07.580074 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.580004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bdf84c6c-t7md7"
Jul 16 22:00:07.717139 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:07.717108    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9bdf84c6c-t7md7"]
Jul 16 22:00:07.719765 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:00:07.719741    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cc69be4_dc26_4fad_a7c4_1b46ecb29369.slice/crio-18827257627a7841e2a836b450986531d95930cf9084820af1221938c507f29f WatchSource:0}: Error finding container 18827257627a7841e2a836b450986531d95930cf9084820af1221938c507f29f: Status 404 returned error can't find the container with id 18827257627a7841e2a836b450986531d95930cf9084820af1221938c507f29f
Jul 16 22:00:08.438730 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:08.438694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bdf84c6c-t7md7" event={"ID":"0cc69be4-dc26-4fad-a7c4-1b46ecb29369","Type":"ContainerStarted","Data":"18827257627a7841e2a836b450986531d95930cf9084820af1221938c507f29f"}
Jul 16 22:00:09.446136 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:09.446097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bdf84c6c-t7md7" event={"ID":"0cc69be4-dc26-4fad-a7c4-1b46ecb29369","Type":"ContainerStarted","Data":"4dc5adbd2591f49f7a76b053cdf69475414c1fae3a811a5f68e20754b023fe95"}
Jul 16 22:00:09.463242 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:09.463191    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9bdf84c6c-t7md7" podStartSLOduration=1.6855257799999999 podStartE2EDuration="2.463173529s" totalImagesPullingTime="777.647749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:00:09.461552845 +0000 UTC m=+675.850826632" watchObservedRunningTime="2026-07-16 22:00:09.463173529 +0000 UTC m=+675.852447315"
Jul 16 22:00:10.042637 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:10.042597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:00:10.148062 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:10.148025    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:00:11.284946 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:11.284892    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:00:11.384398 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:11.384361    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:00:35.333959 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:35.333901    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:00:46.377218 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:46.377135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:00:58.081109 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.081072    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-mx4vs"]
Jul 16 22:00:58.081663 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.081303    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs" podUID="4231a980-36df-4b94-a964-2e5c29048243" containerName="manager" containerID="cri-o://cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7" gracePeriod=30
Jul 16 22:00:58.337459 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.337401    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 22:00:58.345878 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.345857    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgl6w\" (UniqueName: \"kubernetes.io/projected/4231a980-36df-4b94-a964-2e5c29048243-kube-api-access-kgl6w\") pod \"4231a980-36df-4b94-a964-2e5c29048243\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") "
Jul 16 22:00:58.345996 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.345893    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4231a980-36df-4b94-a964-2e5c29048243-webhook-cert\") pod \"4231a980-36df-4b94-a964-2e5c29048243\" (UID: \"4231a980-36df-4b94-a964-2e5c29048243\") "
Jul 16 22:00:58.347962 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.347936    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4231a980-36df-4b94-a964-2e5c29048243-kube-api-access-kgl6w" (OuterVolumeSpecName: "kube-api-access-kgl6w") pod "4231a980-36df-4b94-a964-2e5c29048243" (UID: "4231a980-36df-4b94-a964-2e5c29048243"). InnerVolumeSpecName "kube-api-access-kgl6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:00:58.348057 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.347976    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4231a980-36df-4b94-a964-2e5c29048243-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4231a980-36df-4b94-a964-2e5c29048243" (UID: "4231a980-36df-4b94-a964-2e5c29048243"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:00:58.446873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.446842    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgl6w\" (UniqueName: \"kubernetes.io/projected/4231a980-36df-4b94-a964-2e5c29048243-kube-api-access-kgl6w\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:00:58.446873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.446868    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4231a980-36df-4b94-a964-2e5c29048243-webhook-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:00:58.701468 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.701433    2564 generic.go:356] "Generic (PLEG): container finished" podID="4231a980-36df-4b94-a964-2e5c29048243" containerID="cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7" exitCode=0
Jul 16 22:00:58.701658 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.701483    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs"
Jul 16 22:00:58.701658 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.701478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs" event={"ID":"4231a980-36df-4b94-a964-2e5c29048243","Type":"ContainerDied","Data":"cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7"}
Jul 16 22:00:58.701658 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.701547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-mx4vs" event={"ID":"4231a980-36df-4b94-a964-2e5c29048243","Type":"ContainerDied","Data":"6d44e3fee24f295ff7148582830b3391bace707a66772797a596365ab2bbcd95"}
Jul 16 22:00:58.701658 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.701577    2564 scope.go:117] "RemoveContainer" containerID="cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7"
Jul 16 22:00:58.711258 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.711241    2564 scope.go:117] "RemoveContainer" containerID="cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7"
Jul 16 22:00:58.711509 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:00:58.711493    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7\": container with ID starting with cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7 not found: ID does not exist" containerID="cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7"
Jul 16 22:00:58.711559 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.711519    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7"} err="failed to get container status \"cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7\": rpc error: code = NotFound desc = could not find container \"cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7\": container with ID starting with cf8a9615764cecd4b38ad54748b5415b7b4e44433da4c96ee26fbcaa1c3f9ee7 not found: ID does not exist"
Jul 16 22:00:58.723059 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.723034    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-mx4vs"]
Jul 16 22:00:58.729118 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:58.729097    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-mx4vs"]
Jul 16 22:00:59.077328 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:00:59.077221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:01:00.125416 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:00.125377    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4231a980-36df-4b94-a964-2e5c29048243" path="/var/lib/kubelet/pods/4231a980-36df-4b94-a964-2e5c29048243/volumes"
Jul 16 22:01:59.571121 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.571089    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wz7p9"]
Jul 16 22:01:59.571527 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.571513    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4231a980-36df-4b94-a964-2e5c29048243" containerName="manager"
Jul 16 22:01:59.571575 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.571529    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4231a980-36df-4b94-a964-2e5c29048243" containerName="manager"
Jul 16 22:01:59.571616 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.571608    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4231a980-36df-4b94-a964-2e5c29048243" containerName="manager"
Jul 16 22:01:59.574569 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.574552    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:01:59.577452 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.577431    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d7qf9\""
Jul 16 22:01:59.577809 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.577796    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 22:01:59.577904 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.577883    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 22:01:59.592525 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.592503    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wz7p9"]
Jul 16 22:01:59.674119 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.674085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q26x9\" (UniqueName: \"kubernetes.io/projected/de040b67-e180-40b1-a04a-d2b33255cc69-kube-api-access-q26x9\") pod \"maas-controller-6f4d5cdddc-wz7p9\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") " pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:01:59.674288 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.674147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de040b67-e180-40b1-a04a-d2b33255cc69-webhook-cert\") pod \"maas-controller-6f4d5cdddc-wz7p9\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") " pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:01:59.774710 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.774684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q26x9\" (UniqueName: \"kubernetes.io/projected/de040b67-e180-40b1-a04a-d2b33255cc69-kube-api-access-q26x9\") pod \"maas-controller-6f4d5cdddc-wz7p9\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") " pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:01:59.774858 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.774748    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de040b67-e180-40b1-a04a-d2b33255cc69-webhook-cert\") pod \"maas-controller-6f4d5cdddc-wz7p9\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") " pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:01:59.777335 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.777315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de040b67-e180-40b1-a04a-d2b33255cc69-webhook-cert\") pod \"maas-controller-6f4d5cdddc-wz7p9\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") " pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:01:59.783353 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.783332    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q26x9\" (UniqueName: \"kubernetes.io/projected/de040b67-e180-40b1-a04a-d2b33255cc69-kube-api-access-q26x9\") pod \"maas-controller-6f4d5cdddc-wz7p9\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") " pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:01:59.885378 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:01:59.885292    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:02:00.029960 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:00.029930    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wz7p9"]
Jul 16 22:02:00.032859 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:02:00.032832    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde040b67_e180_40b1_a04a_d2b33255cc69.slice/crio-8b372664cff7475634d2a2758dc4db1f5e555cde44a60847649331e785411345 WatchSource:0}: Error finding container 8b372664cff7475634d2a2758dc4db1f5e555cde44a60847649331e785411345: Status 404 returned error can't find the container with id 8b372664cff7475634d2a2758dc4db1f5e555cde44a60847649331e785411345
Jul 16 22:02:01.007273 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:01.007172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9" event={"ID":"de040b67-e180-40b1-a04a-d2b33255cc69","Type":"ContainerStarted","Data":"8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4"}
Jul 16 22:02:01.007273 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:01.007240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9" event={"ID":"de040b67-e180-40b1-a04a-d2b33255cc69","Type":"ContainerStarted","Data":"8b372664cff7475634d2a2758dc4db1f5e555cde44a60847649331e785411345"}
Jul 16 22:02:01.007691 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:01.007326    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:02:01.042271 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:01.042213    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9" podStartSLOduration=1.382131768 podStartE2EDuration="2.042195363s" totalImagesPullingTime="660.063595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:02:01.039565764 +0000 UTC m=+787.428839549" watchObservedRunningTime="2026-07-16 22:02:01.042195363 +0000 UTC m=+787.431469149"
Jul 16 22:02:03.231632 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:03.231590    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:02:12.015789 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:12.015759    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:02:15.172680 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:15.172589    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:02:16.716271 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:16.716218    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wz7p9"]
Jul 16 22:02:16.716750 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:16.716536    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9" podUID="de040b67-e180-40b1-a04a-d2b33255cc69" containerName="manager" containerID="cri-o://8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4" gracePeriod=30
Jul 16 22:02:16.970438 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:16.970382    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:02:17.017157 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.017123    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de040b67-e180-40b1-a04a-d2b33255cc69-webhook-cert\") pod \"de040b67-e180-40b1-a04a-d2b33255cc69\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") "
Jul 16 22:02:17.017348 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.017249    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q26x9\" (UniqueName: \"kubernetes.io/projected/de040b67-e180-40b1-a04a-d2b33255cc69-kube-api-access-q26x9\") pod \"de040b67-e180-40b1-a04a-d2b33255cc69\" (UID: \"de040b67-e180-40b1-a04a-d2b33255cc69\") "
Jul 16 22:02:17.019430 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.019393    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de040b67-e180-40b1-a04a-d2b33255cc69-kube-api-access-q26x9" (OuterVolumeSpecName: "kube-api-access-q26x9") pod "de040b67-e180-40b1-a04a-d2b33255cc69" (UID: "de040b67-e180-40b1-a04a-d2b33255cc69"). InnerVolumeSpecName "kube-api-access-q26x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:02:17.019688 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.019667    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de040b67-e180-40b1-a04a-d2b33255cc69-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "de040b67-e180-40b1-a04a-d2b33255cc69" (UID: "de040b67-e180-40b1-a04a-d2b33255cc69"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:02:17.091418 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.091385    2564 generic.go:356] "Generic (PLEG): container finished" podID="de040b67-e180-40b1-a04a-d2b33255cc69" containerID="8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4" exitCode=0
Jul 16 22:02:17.091586 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.091426    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9"
Jul 16 22:02:17.091586 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.091457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9" event={"ID":"de040b67-e180-40b1-a04a-d2b33255cc69","Type":"ContainerDied","Data":"8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4"}
Jul 16 22:02:17.091586 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.091500    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-wz7p9" event={"ID":"de040b67-e180-40b1-a04a-d2b33255cc69","Type":"ContainerDied","Data":"8b372664cff7475634d2a2758dc4db1f5e555cde44a60847649331e785411345"}
Jul 16 22:02:17.091586 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.091521    2564 scope.go:117] "RemoveContainer" containerID="8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4"
Jul 16 22:02:17.104657 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.104641    2564 scope.go:117] "RemoveContainer" containerID="8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4"
Jul 16 22:02:17.104902 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:02:17.104885    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4\": container with ID starting with 8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4 not found: ID does not exist" containerID="8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4"
Jul 16 22:02:17.104980 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.104935    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4"} err="failed to get container status \"8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4\": rpc error: code = NotFound desc = could not find container \"8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4\": container with ID starting with 8d3bd9de05b58cdb6eb433305055a870b28f0ee64b2fec60b79438a7eb2a27d4 not found: ID does not exist"
Jul 16 22:02:17.117889 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.117867    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q26x9\" (UniqueName: \"kubernetes.io/projected/de040b67-e180-40b1-a04a-d2b33255cc69-kube-api-access-q26x9\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:02:17.117982 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.117891    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de040b67-e180-40b1-a04a-d2b33255cc69-webhook-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:02:17.121782 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.121759    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wz7p9"]
Jul 16 22:02:17.128215 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.128195    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wz7p9"]
Jul 16 22:02:17.402378 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.402292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:02:17.954646 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.954606    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wqmfp"]
Jul 16 22:02:17.955046 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.955032    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de040b67-e180-40b1-a04a-d2b33255cc69" containerName="manager"
Jul 16 22:02:17.955046 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.955047    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de040b67-e180-40b1-a04a-d2b33255cc69" containerName="manager"
Jul 16 22:02:17.955119 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.955105    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de040b67-e180-40b1-a04a-d2b33255cc69" containerName="manager"
Jul 16 22:02:17.958095 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.958074    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:17.961783 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.961763    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d7qf9\""
Jul 16 22:02:17.961929 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.961762    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 22:02:17.961929 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.961790    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 22:02:17.968389 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:17.968369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wqmfp"]
Jul 16 22:02:18.027379 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.027345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/241531e3-1b0f-492e-a104-d49454633c50-webhook-cert\") pod \"maas-controller-6f4d5cdddc-wqmfp\" (UID: \"241531e3-1b0f-492e-a104-d49454633c50\") " pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:18.027566 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.027418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ml2t\" (UniqueName: \"kubernetes.io/projected/241531e3-1b0f-492e-a104-d49454633c50-kube-api-access-6ml2t\") pod \"maas-controller-6f4d5cdddc-wqmfp\" (UID: \"241531e3-1b0f-492e-a104-d49454633c50\") " pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:18.126446 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.126415    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de040b67-e180-40b1-a04a-d2b33255cc69" path="/var/lib/kubelet/pods/de040b67-e180-40b1-a04a-d2b33255cc69/volumes"
Jul 16 22:02:18.128726 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.128702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ml2t\" (UniqueName: \"kubernetes.io/projected/241531e3-1b0f-492e-a104-d49454633c50-kube-api-access-6ml2t\") pod \"maas-controller-6f4d5cdddc-wqmfp\" (UID: \"241531e3-1b0f-492e-a104-d49454633c50\") " pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:18.128878 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.128769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/241531e3-1b0f-492e-a104-d49454633c50-webhook-cert\") pod \"maas-controller-6f4d5cdddc-wqmfp\" (UID: \"241531e3-1b0f-492e-a104-d49454633c50\") " pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:18.131451 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.131427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/241531e3-1b0f-492e-a104-d49454633c50-webhook-cert\") pod \"maas-controller-6f4d5cdddc-wqmfp\" (UID: \"241531e3-1b0f-492e-a104-d49454633c50\") " pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:18.139377 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.139353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ml2t\" (UniqueName: \"kubernetes.io/projected/241531e3-1b0f-492e-a104-d49454633c50-kube-api-access-6ml2t\") pod \"maas-controller-6f4d5cdddc-wqmfp\" (UID: \"241531e3-1b0f-492e-a104-d49454633c50\") " pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:18.269472 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.269381    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:18.407818 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.407783    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f4d5cdddc-wqmfp"]
Jul 16 22:02:18.411434 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:02:18.411404    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241531e3_1b0f_492e_a104_d49454633c50.slice/crio-fb71595544dedd0155e881aa82bf4a71e0c646bbae2b4d88dff10ffd7bcf3b52 WatchSource:0}: Error finding container fb71595544dedd0155e881aa82bf4a71e0c646bbae2b4d88dff10ffd7bcf3b52: Status 404 returned error can't find the container with id fb71595544dedd0155e881aa82bf4a71e0c646bbae2b4d88dff10ffd7bcf3b52
Jul 16 22:02:18.692589 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:18.692556    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:02:19.104226 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:19.104135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp" event={"ID":"241531e3-1b0f-492e-a104-d49454633c50","Type":"ContainerStarted","Data":"95d5d79c832b2e144436ec9e22025dbe1731ba4c39b22b844e7f4dc68b3600f3"}
Jul 16 22:02:19.104226 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:19.104186    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp" event={"ID":"241531e3-1b0f-492e-a104-d49454633c50","Type":"ContainerStarted","Data":"fb71595544dedd0155e881aa82bf4a71e0c646bbae2b4d88dff10ffd7bcf3b52"}
Jul 16 22:02:19.104727 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:19.104282    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:19.125872 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:19.125816    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp" podStartSLOduration=1.771282874 podStartE2EDuration="2.12579625s" totalImagesPullingTime="354.513376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:02:19.124203326 +0000 UTC m=+805.513477133" watchObservedRunningTime="2026-07-16 22:02:19.12579625 +0000 UTC m=+805.515070037"
Jul 16 22:02:30.111363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:30.111330    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f4d5cdddc-wqmfp"
Jul 16 22:02:53.948946 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:53.948884    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:02:54.126903 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:54.126866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:02:54.251172 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:02:54.251083    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:00.761184 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:00.761147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:00.869469 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:00.869431    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:01.083010 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:01.082905    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:01.252184 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:01.252147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:08.452572 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:08.452523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:08.584529 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:08.584496    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:29.528203 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:29.528161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:03:54.075174 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:54.075146    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:03:54.075616 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:54.075546    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:03:54.081007 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:54.080985    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:03:54.081187 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:03:54.081168    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:05:45.116557 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:05:45.116479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:05:46.298065 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:05:46.298017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:06:14.734786 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:06:14.734751    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:07:05.365451 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:05.365366    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"]
Jul 16 22:07:05.366090 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:05.365598    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw" podUID="703fafd3-e4c4-4eeb-9f83-36656176b344" containerName="manager" containerID="cri-o://fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1" gracePeriod=10
Jul 16 22:07:08.130296 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.130244    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw" podUID="703fafd3-e4c4-4eeb-9f83-36656176b344" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.36:8081/readyz\": dial tcp 10.132.0.36:8081: connect: connection refused"
Jul 16 22:07:08.228713 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.228690    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 22:07:08.263984 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.263956    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fafd3-e4c4-4eeb-9f83-36656176b344-extensions-socket-volume\") pod \"703fafd3-e4c4-4eeb-9f83-36656176b344\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") "
Jul 16 22:07:08.264143 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.264096    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm4xn\" (UniqueName: \"kubernetes.io/projected/703fafd3-e4c4-4eeb-9f83-36656176b344-kube-api-access-tm4xn\") pod \"703fafd3-e4c4-4eeb-9f83-36656176b344\" (UID: \"703fafd3-e4c4-4eeb-9f83-36656176b344\") "
Jul 16 22:07:08.264360 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.264330    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/703fafd3-e4c4-4eeb-9f83-36656176b344-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "703fafd3-e4c4-4eeb-9f83-36656176b344" (UID: "703fafd3-e4c4-4eeb-9f83-36656176b344"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:07:08.266254 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.266227    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703fafd3-e4c4-4eeb-9f83-36656176b344-kube-api-access-tm4xn" (OuterVolumeSpecName: "kube-api-access-tm4xn") pod "703fafd3-e4c4-4eeb-9f83-36656176b344" (UID: "703fafd3-e4c4-4eeb-9f83-36656176b344"). InnerVolumeSpecName "kube-api-access-tm4xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:07:08.364893 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.364818    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm4xn\" (UniqueName: \"kubernetes.io/projected/703fafd3-e4c4-4eeb-9f83-36656176b344-kube-api-access-tm4xn\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:07:08.364893 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.364847    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/703fafd3-e4c4-4eeb-9f83-36656176b344-extensions-socket-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:07:08.600404 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.600370    2564 generic.go:356] "Generic (PLEG): container finished" podID="703fafd3-e4c4-4eeb-9f83-36656176b344" containerID="fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1" exitCode=0
Jul 16 22:07:08.600577 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.600416    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"
Jul 16 22:07:08.600577 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.600458    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw" event={"ID":"703fafd3-e4c4-4eeb-9f83-36656176b344","Type":"ContainerDied","Data":"fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1"}
Jul 16 22:07:08.600577 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.600513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw" event={"ID":"703fafd3-e4c4-4eeb-9f83-36656176b344","Type":"ContainerDied","Data":"8c061020d1d15a95eaccb197c3209e1f5c8ea7e9e0e8f5a8917af19d7a3710d3"}
Jul 16 22:07:08.600577 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.600537    2564 scope.go:117] "RemoveContainer" containerID="fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1"
Jul 16 22:07:08.611422 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.611405    2564 scope.go:117] "RemoveContainer" containerID="fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1"
Jul 16 22:07:08.611674 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:07:08.611653    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1\": container with ID starting with fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1 not found: ID does not exist" containerID="fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1"
Jul 16 22:07:08.611770 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.611678    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1"} err="failed to get container status \"fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1\": rpc error: code = NotFound desc = could not find container \"fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1\": container with ID starting with fb50bf5c14a293b924c2ff360125de60192a4ba0c5a7eb9e7d0a3de88ab598a1 not found: ID does not exist"
Jul 16 22:07:08.623335 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.623285    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"]
Jul 16 22:07:08.627310 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:08.627290    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lbxmw"]
Jul 16 22:07:10.125796 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:07:10.125759    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="703fafd3-e4c4-4eeb-9f83-36656176b344" path="/var/lib/kubelet/pods/703fafd3-e4c4-4eeb-9f83-36656176b344/volumes"
Jul 16 22:08:06.529202 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.529166    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"]
Jul 16 22:08:06.529617 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.529585    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="703fafd3-e4c4-4eeb-9f83-36656176b344" containerName="manager"
Jul 16 22:08:06.529617 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.529597    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="703fafd3-e4c4-4eeb-9f83-36656176b344" containerName="manager"
Jul 16 22:08:06.529689 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.529671    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="703fafd3-e4c4-4eeb-9f83-36656176b344" containerName="manager"
Jul 16 22:08:06.533879 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.533860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.536634 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.536614    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jv95z\""
Jul 16 22:08:06.542268 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.542248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm2s6\" (UniqueName: \"kubernetes.io/projected/1d2e9a4d-2f95-4ffd-8490-ce5a04a90580-kube-api-access-fm2s6\") pod \"kuadrant-operator-controller-manager-86878c769f-592vz\" (UID: \"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.542373 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.542297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d2e9a4d-2f95-4ffd-8490-ce5a04a90580-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-592vz\" (UID: \"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.545734 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.545709    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"]
Jul 16 22:08:06.643739 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.643702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm2s6\" (UniqueName: \"kubernetes.io/projected/1d2e9a4d-2f95-4ffd-8490-ce5a04a90580-kube-api-access-fm2s6\") pod \"kuadrant-operator-controller-manager-86878c769f-592vz\" (UID: \"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.643945 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.643760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d2e9a4d-2f95-4ffd-8490-ce5a04a90580-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-592vz\" (UID: \"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.644114 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.644094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d2e9a4d-2f95-4ffd-8490-ce5a04a90580-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-592vz\" (UID: \"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.653002 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.652982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm2s6\" (UniqueName: \"kubernetes.io/projected/1d2e9a4d-2f95-4ffd-8490-ce5a04a90580-kube-api-access-fm2s6\") pod \"kuadrant-operator-controller-manager-86878c769f-592vz\" (UID: \"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.844494 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.844414    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:06.989095 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:06.989041    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"]
Jul 16 22:08:06.993488 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:08:06.993456    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2e9a4d_2f95_4ffd_8490_ce5a04a90580.slice/crio-13b8ce803e0213029ef2be188b874bf263c767883a937f5c966ef59ea0ac08a4 WatchSource:0}: Error finding container 13b8ce803e0213029ef2be188b874bf263c767883a937f5c966ef59ea0ac08a4: Status 404 returned error can't find the container with id 13b8ce803e0213029ef2be188b874bf263c767883a937f5c966ef59ea0ac08a4
Jul 16 22:08:07.893852 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:07.893817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz" event={"ID":"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580","Type":"ContainerStarted","Data":"dc50ddab420115042d90fecbb00bb5537da1d3011f5ea2b2ddcfd3721fd73a77"}
Jul 16 22:08:07.894249 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:07.893936    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:07.894249 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:07.893962    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz" event={"ID":"1d2e9a4d-2f95-4ffd-8490-ce5a04a90580","Type":"ContainerStarted","Data":"13b8ce803e0213029ef2be188b874bf263c767883a937f5c966ef59ea0ac08a4"}
Jul 16 22:08:07.914844 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:07.914796    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz" podStartSLOduration=1.914781845 podStartE2EDuration="1.914781845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:08:07.912790501 +0000 UTC m=+1154.302064297" watchObservedRunningTime="2026-07-16 22:08:07.914781845 +0000 UTC m=+1154.304055632"
Jul 16 22:08:18.900981 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:18.900948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-592vz"
Jul 16 22:08:54.105984 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:54.105956    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:08:54.107776 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:54.107744    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:08:54.111798 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:54.111780    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:08:54.113322 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:08:54.113305    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:09:53.183786 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:09:53.183705    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:09:53.268964 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:09:53.268898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:10:27.902841 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:10:27.902800    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:13:17.369486 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:13:17.369443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:13:18.561477 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:13:18.561422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:13:54.138464 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:13:54.138437    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:13:54.140736 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:13:54.140709    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:13:54.143965 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:13:54.143941    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:13:54.146087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:13:54.146069    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:14:06.592054 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:06.592015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:18.424750 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:18.424616    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:26.860401 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:26.860350    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:31.903893 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:31.903828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:33.569014 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:33.568954    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:35.080719 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:35.080679    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:35.209949 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:35.209880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:35.664154 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:35.664119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:42.587839 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:42.587797    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:42.766110 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:42.766072    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:43.077987 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:43.077956    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:43.296833 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:43.296797    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:44.132927 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.132880    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:44.280938 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.280883    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:44.286126 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.286100    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"]
Jul 16 22:14:44.289626 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.289610    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.292813 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.292796    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d3f2b144-serving-cert\""
Jul 16 22:14:44.336835 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.336805    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"]
Jul 16 22:14:44.400103 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.400029    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-openshift-service-ca\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.400103 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.400091    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lrc\" (UniqueName: \"kubernetes.io/projected/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-kube-api-access-h2lrc\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.400293 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.400143    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.501216 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.501184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-openshift-service-ca\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.501416 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.501245    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lrc\" (UniqueName: \"kubernetes.io/projected/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-kube-api-access-h2lrc\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.501416 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.501282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.501416 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:14:44.501408    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-serving-cert: secret "maas-api-e2e-ait-d3f2b144-serving-cert" not found
Jul 16 22:14:44.501611 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:14:44.501480    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls podName:5d3c8fc2-0f00-48a8-bea1-680e679dfc8e nodeName:}" failed. No retries permitted until 2026-07-16 22:14:45.001464656 +0000 UTC m=+1551.390738421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls") pod "maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" (UID: "5d3c8fc2-0f00-48a8-bea1-680e679dfc8e") : secret "maas-api-e2e-ait-d3f2b144-serving-cert" not found
Jul 16 22:14:44.501964 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.501901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-openshift-service-ca\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.514864 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.514840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:44.519200 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.519175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lrc\" (UniqueName: \"kubernetes.io/projected/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-kube-api-access-h2lrc\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:44.655427 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:44.655341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:45.006443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.006398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:45.009200 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.009171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls\") pod \"maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:45.180656 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.180624    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:45.199388 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.199361    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:45.338166 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.338132    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:45.352468 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.352445    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"]
Jul 16 22:14:45.355075 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:14:45.355049    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3c8fc2_0f00_48a8_bea1_680e679dfc8e.slice/crio-be2f19fbf90ea5e7501c34b6ea2cee9aaff53274f04e8bb8845e101d7d26552f WatchSource:0}: Error finding container be2f19fbf90ea5e7501c34b6ea2cee9aaff53274f04e8bb8845e101d7d26552f: Status 404 returned error can't find the container with id be2f19fbf90ea5e7501c34b6ea2cee9aaff53274f04e8bb8845e101d7d26552f
Jul 16 22:14:45.356410 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.356390    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:14:45.939126 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:45.939087    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerStarted","Data":"be2f19fbf90ea5e7501c34b6ea2cee9aaff53274f04e8bb8845e101d7d26552f"}
Jul 16 22:14:46.945491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:46.945450    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerID="8f519b896955418ebe621920a7cb1f4ef64c7411a4087ab1ff0c180bc38690ea" exitCode=1
Jul 16 22:14:46.945895 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:46.945546    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerDied","Data":"8f519b896955418ebe621920a7cb1f4ef64c7411a4087ab1ff0c180bc38690ea"}
Jul 16 22:14:46.945895 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:46.945758    2564 scope.go:117] "RemoveContainer" containerID="8f519b896955418ebe621920a7cb1f4ef64c7411a4087ab1ff0c180bc38690ea"
Jul 16 22:14:47.952560 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:47.952521    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerID="8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5" exitCode=1
Jul 16 22:14:47.952987 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:47.952584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerDied","Data":"8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5"}
Jul 16 22:14:47.952987 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:47.952642    2564 scope.go:117] "RemoveContainer" containerID="8f519b896955418ebe621920a7cb1f4ef64c7411a4087ab1ff0c180bc38690ea"
Jul 16 22:14:47.952987 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:47.952859    2564 scope.go:117] "RemoveContainer" containerID="8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5"
Jul 16 22:14:47.953153 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:14:47.953127    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:14:48.142094 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:48.142027    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:48.960814 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:48.960790    2564 scope.go:117] "RemoveContainer" containerID="8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5"
Jul 16 22:14:48.961223 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:14:48.960997    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:14:50.200138 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:50.200106    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:50.200510 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:50.200445    2564 scope.go:117] "RemoveContainer" containerID="8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5"
Jul 16 22:14:50.200652 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:14:50.200632    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:14:55.200123 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:55.200075    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:14:55.200526 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:14:55.200476    2564 scope.go:117] "RemoveContainer" containerID="8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5"
Jul 16 22:14:55.200668 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:14:55.200648    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:15:00.154747 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.154716    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"]
Jul 16 22:15:00.158067 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.158042    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"
Jul 16 22:15:00.159861 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.159835    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k"]
Jul 16 22:15:00.160695 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.160662    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:15:00.163254 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.163234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k"
Jul 16 22:15:00.167403 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.167386    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"
Jul 16 22:15:00.175580 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.175544    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"]
Jul 16 22:15:00.176363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.176342    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k"
Jul 16 22:15:00.178295 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.178272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k"]
Jul 16 22:15:00.368747 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.368720    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"]
Jul 16 22:15:00.370145 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:15:00.370120    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e6e520_2a5d_4698_b58e_aba35e8bf126.slice/crio-4a60c51fea505d7dcfabb14bceb3d79e8fd5b26becd58ebe3194d91b83ac1158 WatchSource:0}: Error finding container 4a60c51fea505d7dcfabb14bceb3d79e8fd5b26becd58ebe3194d91b83ac1158: Status 404 returned error can't find the container with id 4a60c51fea505d7dcfabb14bceb3d79e8fd5b26becd58ebe3194d91b83ac1158
Jul 16 22:15:00.562692 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:00.562669    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k"]
Jul 16 22:15:00.564257 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:15:00.564234    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70f81a2a_db21_44fe_afc3_fa55f28fc1d9.slice/crio-070fd6fa0bfa805a381ef7bb7541f9c9e4c6518b6761481b4303b30715acdb5e WatchSource:0}: Error finding container 070fd6fa0bfa805a381ef7bb7541f9c9e4c6518b6761481b4303b30715acdb5e: Status 404 returned error can't find the container with id 070fd6fa0bfa805a381ef7bb7541f9c9e4c6518b6761481b4303b30715acdb5e
Jul 16 22:15:01.024975 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:01.024848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k" event={"ID":"70f81a2a-db21-44fe-afc3-fa55f28fc1d9","Type":"ContainerStarted","Data":"8073397aacba5a3dde005308421e16d19a2f7d40481cb8fe8d03fda218d920d9"}
Jul 16 22:15:01.024975 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:01.024934    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k" event={"ID":"70f81a2a-db21-44fe-afc3-fa55f28fc1d9","Type":"ContainerStarted","Data":"070fd6fa0bfa805a381ef7bb7541f9c9e4c6518b6761481b4303b30715acdb5e"}
Jul 16 22:15:01.026832 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:01.026553    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" event={"ID":"08e6e520-2a5d-4698-b58e-aba35e8bf126","Type":"ContainerStarted","Data":"987b4092490a92f794907fb42a520c133ca89d20243f9d1d46da27dd2e63ad44"}
Jul 16 22:15:01.026832 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:01.026601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" event={"ID":"08e6e520-2a5d-4698-b58e-aba35e8bf126","Type":"ContainerStarted","Data":"4a60c51fea505d7dcfabb14bceb3d79e8fd5b26becd58ebe3194d91b83ac1158"}
Jul 16 22:15:01.045921 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:01.045859    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k" podStartSLOduration=1.045844451 podStartE2EDuration="1.045844451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:01.044321752 +0000 UTC m=+1567.433595549" watchObservedRunningTime="2026-07-16 22:15:01.045844451 +0000 UTC m=+1567.435118238"
Jul 16 22:15:01.062893 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:01.062845    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" podStartSLOduration=1.062830529 podStartE2EDuration="1.062830529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:01.061730557 +0000 UTC m=+1567.451004345" watchObservedRunningTime="2026-07-16 22:15:01.062830529 +0000 UTC m=+1567.452104315"
Jul 16 22:15:02.034632 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:02.034599    2564 generic.go:356] "Generic (PLEG): container finished" podID="70f81a2a-db21-44fe-afc3-fa55f28fc1d9" containerID="8073397aacba5a3dde005308421e16d19a2f7d40481cb8fe8d03fda218d920d9" exitCode=0
Jul 16 22:15:02.035064 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:02.034685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k" event={"ID":"70f81a2a-db21-44fe-afc3-fa55f28fc1d9","Type":"ContainerDied","Data":"8073397aacba5a3dde005308421e16d19a2f7d40481cb8fe8d03fda218d920d9"}
Jul 16 22:15:02.036374 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:02.036343    2564 generic.go:356] "Generic (PLEG): container finished" podID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerID="987b4092490a92f794907fb42a520c133ca89d20243f9d1d46da27dd2e63ad44" exitCode=7
Jul 16 22:15:02.036515 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:02.036390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" event={"ID":"08e6e520-2a5d-4698-b58e-aba35e8bf126","Type":"ContainerDied","Data":"987b4092490a92f794907fb42a520c133ca89d20243f9d1d46da27dd2e63ad44"}
Jul 16 22:15:02.036595 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:02.036578    2564 scope.go:117] "RemoveContainer" containerID="987b4092490a92f794907fb42a520c133ca89d20243f9d1d46da27dd2e63ad44"
Jul 16 22:15:03.043025 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:03.042930    2564 generic.go:356] "Generic (PLEG): container finished" podID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerID="ba5fc2b955cf4630f1cc08c9f839d05c5b2e3b62f781fd4e1a5765276468d6f9" exitCode=7
Jul 16 22:15:03.043025 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:03.043006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" event={"ID":"08e6e520-2a5d-4698-b58e-aba35e8bf126","Type":"ContainerDied","Data":"ba5fc2b955cf4630f1cc08c9f839d05c5b2e3b62f781fd4e1a5765276468d6f9"}
Jul 16 22:15:03.043491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:03.043067    2564 scope.go:117] "RemoveContainer" containerID="987b4092490a92f794907fb42a520c133ca89d20243f9d1d46da27dd2e63ad44"
Jul 16 22:15:03.043491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:03.043262    2564 scope.go:117] "RemoveContainer" containerID="ba5fc2b955cf4630f1cc08c9f839d05c5b2e3b62f781fd4e1a5765276468d6f9"
Jul 16 22:15:03.043890 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:03.043490    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946_odh-ai-gateway-infra(08e6e520-2a5d-4698-b58e-aba35e8bf126)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126"
Jul 16 22:15:03.186033 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:03.186010    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k"
Jul 16 22:15:04.049561 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:04.049530    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k"
Jul 16 22:15:04.050001 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:04.049537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737335-f8v8k" event={"ID":"70f81a2a-db21-44fe-afc3-fa55f28fc1d9","Type":"ContainerDied","Data":"070fd6fa0bfa805a381ef7bb7541f9c9e4c6518b6761481b4303b30715acdb5e"}
Jul 16 22:15:04.050001 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:04.049659    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="070fd6fa0bfa805a381ef7bb7541f9c9e4c6518b6761481b4303b30715acdb5e"
Jul 16 22:15:04.051362 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:04.051346    2564 scope.go:117] "RemoveContainer" containerID="ba5fc2b955cf4630f1cc08c9f839d05c5b2e3b62f781fd4e1a5765276468d6f9"
Jul 16 22:15:04.051552 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:04.051535    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946_odh-ai-gateway-infra(08e6e520-2a5d-4698-b58e-aba35e8bf126)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126"
Jul 16 22:15:10.120180 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:10.120144    2564 scope.go:117] "RemoveContainer" containerID="8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5"
Jul 16 22:15:11.087563 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:11.087527    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerID="bc5122112233571d9ff0bdaf59432ee08381c7fab307f8d194954278c3fc6a2b" exitCode=1
Jul 16 22:15:11.087700 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:11.087592    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerDied","Data":"bc5122112233571d9ff0bdaf59432ee08381c7fab307f8d194954278c3fc6a2b"}
Jul 16 22:15:11.087700 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:11.087646    2564 scope.go:117] "RemoveContainer" containerID="8e3c09289ee6b1b0273faa9efea0b1e76fca70080c5fe7e37593daf7505a63d5"
Jul 16 22:15:11.088029 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:11.088003    2564 scope.go:117] "RemoveContainer" containerID="bc5122112233571d9ff0bdaf59432ee08381c7fab307f8d194954278c3fc6a2b"
Jul 16 22:15:11.088257 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:11.088236    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:15:15.200089 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:15.200042    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:15:15.200500 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:15.200142    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:15:15.200500 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:15.200473    2564 scope.go:117] "RemoveContainer" containerID="bc5122112233571d9ff0bdaf59432ee08381c7fab307f8d194954278c3fc6a2b"
Jul 16 22:15:15.200658 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:15.200637    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:15:18.119925 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:18.119889    2564 scope.go:117] "RemoveContainer" containerID="ba5fc2b955cf4630f1cc08c9f839d05c5b2e3b62f781fd4e1a5765276468d6f9"
Jul 16 22:15:19.131691 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:19.131607    2564 generic.go:356] "Generic (PLEG): container finished" podID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerID="12ce904ae08e4da17cfeec10758f00707d573d30f7776aebb5f98ac1586fbf5b" exitCode=7
Jul 16 22:15:19.131691 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:19.131654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" event={"ID":"08e6e520-2a5d-4698-b58e-aba35e8bf126","Type":"ContainerDied","Data":"12ce904ae08e4da17cfeec10758f00707d573d30f7776aebb5f98ac1586fbf5b"}
Jul 16 22:15:19.132169 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:19.131700    2564 scope.go:117] "RemoveContainer" containerID="ba5fc2b955cf4630f1cc08c9f839d05c5b2e3b62f781fd4e1a5765276468d6f9"
Jul 16 22:15:19.132169 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:19.131956    2564 scope.go:117] "RemoveContainer" containerID="12ce904ae08e4da17cfeec10758f00707d573d30f7776aebb5f98ac1586fbf5b"
Jul 16 22:15:19.132282 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:19.132172    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946_odh-ai-gateway-infra(08e6e520-2a5d-4698-b58e-aba35e8bf126)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126"
Jul 16 22:15:20.171665 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:20.171631    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"]
Jul 16 22:15:20.311183 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:20.311159    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"
Jul 16 22:15:21.150983 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:21.150952    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"
Jul 16 22:15:21.151187 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:21.150980    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946" event={"ID":"08e6e520-2a5d-4698-b58e-aba35e8bf126","Type":"ContainerDied","Data":"4a60c51fea505d7dcfabb14bceb3d79e8fd5b26becd58ebe3194d91b83ac1158"}
Jul 16 22:15:21.151187 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:21.151047    2564 scope.go:117] "RemoveContainer" containerID="12ce904ae08e4da17cfeec10758f00707d573d30f7776aebb5f98ac1586fbf5b"
Jul 16 22:15:21.176646 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:21.176618    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"]
Jul 16 22:15:21.182680 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:21.182656    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d3f2b144-29737335-fp946"]
Jul 16 22:15:22.125815 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:22.125786    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" path="/var/lib/kubelet/pods/08e6e520-2a5d-4698-b58e-aba35e8bf126/volumes"
Jul 16 22:15:26.119969 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:26.119902    2564 scope.go:117] "RemoveContainer" containerID="bc5122112233571d9ff0bdaf59432ee08381c7fab307f8d194954278c3fc6a2b"
Jul 16 22:15:26.120366 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:26.120099    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:15:37.119696 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:37.119665    2564 scope.go:117] "RemoveContainer" containerID="bc5122112233571d9ff0bdaf59432ee08381c7fab307f8d194954278c3fc6a2b"
Jul 16 22:15:38.244448 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:38.244419    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerStarted","Data":"c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"}
Jul 16 22:15:38.244796 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:38.244620    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:15:38.266083 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:38.266036    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podStartSLOduration=51.742458237 podStartE2EDuration="54.266006251s" totalImagesPullingTime="2.523548014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:14:44 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:38.265463737 +0000 UTC m=+1604.654737522" watchObservedRunningTime="2026-07-16 22:15:38.266006251 +0000 UTC m=+1604.655280037"
Jul 16 22:15:39.251535 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:39.251502    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8" exitCode=1
Jul 16 22:15:39.251959 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:39.251592    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerDied","Data":"c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"}
Jul 16 22:15:39.251959 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:39.251654    2564 scope.go:117] "RemoveContainer" containerID="bc5122112233571d9ff0bdaf59432ee08381c7fab307f8d194954278c3fc6a2b"
Jul 16 22:15:39.251959 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:39.251890    2564 scope.go:117] "RemoveContainer" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"
Jul 16 22:15:39.252150 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:39.252128    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:15:40.258055 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:40.258032    2564 scope.go:117] "RemoveContainer" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"
Jul 16 22:15:40.258444 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:40.258194    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:15:45.199736 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:45.199650    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:15:45.200131 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:45.200048    2564 scope.go:117] "RemoveContainer" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"
Jul 16 22:15:45.200245 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:45.200224    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:15:57.119682 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:15:57.119641    2564 scope.go:117] "RemoveContainer" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"
Jul 16 22:15:57.120217 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:15:57.119859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:16:08.120031 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:08.119996    2564 scope.go:117] "RemoveContainer" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"
Jul 16 22:16:08.120500 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:16:08.120237    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:16:19.119551 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:19.119518    2564 scope.go:117] "RemoveContainer" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"
Jul 16 22:16:20.462881 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:20.462851    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f" exitCode=1
Jul 16 22:16:20.463363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:20.462941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerDied","Data":"1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"}
Jul 16 22:16:20.463363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:20.463005    2564 scope.go:117] "RemoveContainer" containerID="c29301e8414aa6648ce25a3d207748e8565b8541276819a354be2e7c94602cf8"
Jul 16 22:16:20.463363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:20.463359    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:16:20.463599 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:16:20.463577    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:16:23.245538 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:23.245503    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:16:23.246012 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:23.245880    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:16:23.246110 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:16:23.246090    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:16:25.200254 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:25.200212    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:16:25.200756 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:25.200733    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:16:25.201002 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:16:25.200974    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:16:38.119328 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:38.119295    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:16:38.119811 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:16:38.119482    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:16:50.119873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:16:50.119841    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:16:50.120387 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:16:50.120037    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:17:05.120137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:05.120105    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:17:05.120543 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:17:05.120277    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:17:20.120326 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:20.120248    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:17:20.120727 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:17:20.120406    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:17:33.119387 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:33.119353    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:17:33.119957 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:17:33.119600    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:17:40.715393 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:40.715357    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:40.799406 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:40.799374    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:45.396089 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:45.396057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:45.458185 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:45.458149    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:46.119920 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:46.119894    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:17:46.894266 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:46.894188    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerStarted","Data":"4ac96b2b7311babc48e58d8dce4ebbcbf2cd08571466ff8cd7b31b869af3ceb0"}
Jul 16 22:17:46.894657 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:46.894395    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:17:47.901109 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:47.901076    2564 generic.go:356] "Generic (PLEG): container finished" podID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerID="4ac96b2b7311babc48e58d8dce4ebbcbf2cd08571466ff8cd7b31b869af3ceb0" exitCode=1
Jul 16 22:17:47.901538 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:47.901162    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerDied","Data":"4ac96b2b7311babc48e58d8dce4ebbcbf2cd08571466ff8cd7b31b869af3ceb0"}
Jul 16 22:17:47.901538 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:47.901219    2564 scope.go:117] "RemoveContainer" containerID="1adadd7b6532b4578c4463c4eb5ff0c442e01589c9e8386a6589dd275834bb4f"
Jul 16 22:17:47.901538 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:47.901349    2564 scope.go:117] "RemoveContainer" containerID="4ac96b2b7311babc48e58d8dce4ebbcbf2cd08571466ff8cd7b31b869af3ceb0"
Jul 16 22:17:47.901678 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:17:47.901549    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:17:48.907873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:48.907849    2564 scope.go:117] "RemoveContainer" containerID="4ac96b2b7311babc48e58d8dce4ebbcbf2cd08571466ff8cd7b31b869af3ceb0"
Jul 16 22:17:48.908272 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:17:48.908056    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm_odh-ai-gateway-infra(5d3c8fc2-0f00-48a8-bea1-680e679dfc8e)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"
Jul 16 22:17:49.355454 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:49.355424    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:49.468620 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:49.468591    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:54.538795 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.538648    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"]
Jul 16 22:17:54.565239 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.565202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:54.689811 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.689786    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:17:54.754333 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.754300    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2lrc\" (UniqueName: \"kubernetes.io/projected/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-kube-api-access-h2lrc\") pod \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") "
Jul 16 22:17:54.754494 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.754413    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-openshift-service-ca\") pod \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") "
Jul 16 22:17:54.754494 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.754434    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls\") pod \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\" (UID: \"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e\") "
Jul 16 22:17:54.754804 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.754781    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" (UID: "5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:17:54.756609 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.756586    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" (UID: "5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:17:54.756723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.756640    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-kube-api-access-h2lrc" (OuterVolumeSpecName: "kube-api-access-h2lrc") pod "5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" (UID: "5d3c8fc2-0f00-48a8-bea1-680e679dfc8e"). InnerVolumeSpecName "kube-api-access-h2lrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:17:54.855822 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.855742    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2lrc\" (UniqueName: \"kubernetes.io/projected/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-kube-api-access-h2lrc\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:17:54.855822 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.855774    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:17:54.855822 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.855784    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:17:54.940569 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.940541    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"
Jul 16 22:17:54.940748 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.940541    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm" event={"ID":"5d3c8fc2-0f00-48a8-bea1-680e679dfc8e","Type":"ContainerDied","Data":"be2f19fbf90ea5e7501c34b6ea2cee9aaff53274f04e8bb8845e101d7d26552f"}
Jul 16 22:17:54.940748 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.940677    2564 scope.go:117] "RemoveContainer" containerID="4ac96b2b7311babc48e58d8dce4ebbcbf2cd08571466ff8cd7b31b869af3ceb0"
Jul 16 22:17:54.963334 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.963309    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"]
Jul 16 22:17:54.969803 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:54.969777    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d3f2b144-55ddc68bc9-hvffm"]
Jul 16 22:17:55.937380 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:55.937343    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:56.127739 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:17:56.127695    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" path="/var/lib/kubelet/pods/5d3c8fc2-0f00-48a8-bea1-680e679dfc8e/volumes"
Jul 16 22:18:01.192546 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:01.192514    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:01.804208 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:01.804170    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:01.913656 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:01.913626    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:02.090948 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:02.090806    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:02.193758 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:02.193723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:02.321984 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:02.321940    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:02.423931 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:02.423823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:03.575210 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.575179    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:03.777950 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.777894    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"]
Jul 16 22:18:03.778518 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778498    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778524    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778539    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.778568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778548    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.778568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778560    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.778568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778569    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778581    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778589    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778604    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778612    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778629    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f81a2a-db21-44fe-afc3-fa55f28fc1d9" containerName="cleanup"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778637    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f81a2a-db21-44fe-afc3-fa55f28fc1d9" containerName="cleanup"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778648    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778656    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778669    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778677    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778780    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f81a2a-db21-44fe-afc3-fa55f28fc1d9" containerName="cleanup"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778793    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778803    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.778807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778812    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.779247 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778819    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.779247 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778830    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.779247 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778838    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:18:03.779247 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.778848    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e6e520-2a5d-4698-b58e-aba35e8bf126" containerName="cleanup"
Jul 16 22:18:03.783631 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.783609    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:03.786486 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.786425    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-39074f9f-serving-cert\""
Jul 16 22:18:03.789242 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.789216    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"]
Jul 16 22:18:03.816750 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.816723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:03.936337 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.936260    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjf2x\" (UniqueName: \"kubernetes.io/projected/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-kube-api-access-kjf2x\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:03.936337 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.936326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-maas-api-tls\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:03.936516 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.936346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-openshift-service-ca\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:03.961014 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:03.960986    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:04.037391 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.037361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-maas-api-tls\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:04.037529 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.037398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-openshift-service-ca\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:04.037529 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.037448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjf2x\" (UniqueName: \"kubernetes.io/projected/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-kube-api-access-kjf2x\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:04.038225 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.038153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-openshift-service-ca\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:04.040078 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.040054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-maas-api-tls\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:04.046550 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.046521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjf2x\" (UniqueName: \"kubernetes.io/projected/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-kube-api-access-kjf2x\") pod \"maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:04.096605 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.096576    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:04.241949 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.241904    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"]
Jul 16 22:18:04.244978 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:18:04.244887    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0eaded_76c9_47cc_bdbf_18fd7972a1ce.slice/crio-8a80dbe82dc42e6e1131e723d8ef4c9d113c73ce126c49c8171aec76a3f0a5ec WatchSource:0}: Error finding container 8a80dbe82dc42e6e1131e723d8ef4c9d113c73ce126c49c8171aec76a3f0a5ec: Status 404 returned error can't find the container with id 8a80dbe82dc42e6e1131e723d8ef4c9d113c73ce126c49c8171aec76a3f0a5ec
Jul 16 22:18:04.286897 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.286867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:04.386579 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:04.386542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:05.001686 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:05.001654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" event={"ID":"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce","Type":"ContainerStarted","Data":"aa27b734deea38f0692864c5fc739e1f7ad249de51cb7b7a26d218d63bf618fc"}
Jul 16 22:18:05.002190 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:05.001709    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" event={"ID":"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce","Type":"ContainerStarted","Data":"8a80dbe82dc42e6e1131e723d8ef4c9d113c73ce126c49c8171aec76a3f0a5ec"}
Jul 16 22:18:05.002190 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:05.001740    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:05.017988 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:05.017937    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podStartSLOduration=1.432418322 podStartE2EDuration="2.017905892s" totalImagesPullingTime="585.48757ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:05.016905606 +0000 UTC m=+1751.406179392" watchObservedRunningTime="2026-07-16 22:18:05.017905892 +0000 UTC m=+1751.407179686"
Jul 16 22:18:06.002178 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:06.002146    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:06.007941 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:06.007896    2564 generic.go:356] "Generic (PLEG): container finished" podID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerID="aa27b734deea38f0692864c5fc739e1f7ad249de51cb7b7a26d218d63bf618fc" exitCode=1
Jul 16 22:18:06.008081 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:06.007969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" event={"ID":"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce","Type":"ContainerDied","Data":"aa27b734deea38f0692864c5fc739e1f7ad249de51cb7b7a26d218d63bf618fc"}
Jul 16 22:18:06.008282 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:06.008267    2564 scope.go:117] "RemoveContainer" containerID="aa27b734deea38f0692864c5fc739e1f7ad249de51cb7b7a26d218d63bf618fc"
Jul 16 22:18:07.015277 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:07.015194    2564 generic.go:356] "Generic (PLEG): container finished" podID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerID="38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd" exitCode=1
Jul 16 22:18:07.015277 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:07.015254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" event={"ID":"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce","Type":"ContainerDied","Data":"38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd"}
Jul 16 22:18:07.015720 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:07.015298    2564 scope.go:117] "RemoveContainer" containerID="aa27b734deea38f0692864c5fc739e1f7ad249de51cb7b7a26d218d63bf618fc"
Jul 16 22:18:07.015720 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:07.015510    2564 scope.go:117] "RemoveContainer" containerID="38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd"
Jul 16 22:18:07.015720 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:07.015711    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:08.029251 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:08.029219    2564 scope.go:117] "RemoveContainer" containerID="38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd"
Jul 16 22:18:08.029626 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:08.029430    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:10.002162 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:10.002129    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:10.002543 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:10.002481    2564 scope.go:117] "RemoveContainer" containerID="38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd"
Jul 16 22:18:10.002658 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:10.002637    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:14.096778 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:14.096745    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:14.097237 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:14.097148    2564 scope.go:117] "RemoveContainer" containerID="38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd"
Jul 16 22:18:14.097338 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:14.097318    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:29.119444 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:29.119415    2564 scope.go:117] "RemoveContainer" containerID="38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd"
Jul 16 22:18:30.142787 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:30.142706    2564 generic.go:356] "Generic (PLEG): container finished" podID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerID="22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb" exitCode=1
Jul 16 22:18:30.142787 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:30.142773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" event={"ID":"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce","Type":"ContainerDied","Data":"22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb"}
Jul 16 22:18:30.143288 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:30.142832    2564 scope.go:117] "RemoveContainer" containerID="38f13f68211e459abb876a178e945fe35e6612e8d746380c1995b8b8b580e6dd"
Jul 16 22:18:30.143288 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:30.143214    2564 scope.go:117] "RemoveContainer" containerID="22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb"
Jul 16 22:18:30.143402 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:30.143380    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:34.097325 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:34.097294    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:34.097705 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:34.097642    2564 scope.go:117] "RemoveContainer" containerID="22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb"
Jul 16 22:18:34.097832 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:34.097813    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:35.002786 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:35.002737    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:18:35.003243 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:35.003225    2564 scope.go:117] "RemoveContainer" containerID="22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb"
Jul 16 22:18:35.003464 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:35.003440    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:46.119761 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:46.119683    2564 scope.go:117] "RemoveContainer" containerID="22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb"
Jul 16 22:18:46.120168 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:18:46.119901    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:18:48.598798 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:48.598759    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:54.167463 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:54.167432    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:18:54.171345 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:54.171321    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:18:54.173145 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:54.173124    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:18:54.176958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:18:54.176934    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:19:01.119639 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:01.119606    2564 scope.go:117] "RemoveContainer" containerID="22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb"
Jul 16 22:19:02.302900 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:02.302808    2564 generic.go:356] "Generic (PLEG): container finished" podID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerID="88feb8fabc24bac7bb24002313b8c9241562bd8c20cf4744bb0ef0da26a51d11" exitCode=1
Jul 16 22:19:02.303362 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:02.302886    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" event={"ID":"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce","Type":"ContainerDied","Data":"88feb8fabc24bac7bb24002313b8c9241562bd8c20cf4744bb0ef0da26a51d11"}
Jul 16 22:19:02.303362 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:02.302990    2564 scope.go:117] "RemoveContainer" containerID="22ca13df2a883fbf1668c7871f2c74db6be5874a0eff9a3f7a4eecb222a340bb"
Jul 16 22:19:02.303362 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:02.303340    2564 scope.go:117] "RemoveContainer" containerID="88feb8fabc24bac7bb24002313b8c9241562bd8c20cf4744bb0ef0da26a51d11"
Jul 16 22:19:02.303575 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:02.303553    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:19:04.097306 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:04.097255    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:19:04.097683 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:04.097644    2564 scope.go:117] "RemoveContainer" containerID="88feb8fabc24bac7bb24002313b8c9241562bd8c20cf4744bb0ef0da26a51d11"
Jul 16 22:19:04.097842 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:04.097821    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:19:05.002827 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:05.002791    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:19:05.003206 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:05.003191    2564 scope.go:117] "RemoveContainer" containerID="88feb8fabc24bac7bb24002313b8c9241562bd8c20cf4744bb0ef0da26a51d11"
Jul 16 22:19:05.003386 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:05.003367    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww_odh-ai-gateway-infra(7d0eaded-76c9-47cc-bdbf-18fd7972a1ce)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"
Jul 16 22:19:08.420615 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:08.420577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:08.534998 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:08.534959    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:15.624455 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.624422    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"]
Jul 16 22:19:15.684861 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.684831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:15.773216 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.773193    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:19:15.848025 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.847984    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:15.874274 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.874244    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-openshift-service-ca\") pod \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") "
Jul 16 22:19:15.874441 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.874332    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjf2x\" (UniqueName: \"kubernetes.io/projected/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-kube-api-access-kjf2x\") pod \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") "
Jul 16 22:19:15.874441 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.874429    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-maas-api-tls\") pod \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\" (UID: \"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce\") "
Jul 16 22:19:15.874619 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.874564    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" (UID: "7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:15.874753 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.874736    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:19:15.876473 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.876447    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-kube-api-access-kjf2x" (OuterVolumeSpecName: "kube-api-access-kjf2x") pod "7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" (UID: "7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"). InnerVolumeSpecName "kube-api-access-kjf2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:19:15.876571 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.876532    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" (UID: "7d0eaded-76c9-47cc-bdbf-18fd7972a1ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:15.976154 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.976120    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:19:15.976154 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:15.976147    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjf2x\" (UniqueName: \"kubernetes.io/projected/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce-kube-api-access-kjf2x\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:19:16.083596 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:16.083563    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:16.377301 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:16.377267    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"
Jul 16 22:19:16.377474 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:16.377302    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww" event={"ID":"7d0eaded-76c9-47cc-bdbf-18fd7972a1ce","Type":"ContainerDied","Data":"8a80dbe82dc42e6e1131e723d8ef4c9d113c73ce126c49c8171aec76a3f0a5ec"}
Jul 16 22:19:16.377474 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:16.377360    2564 scope.go:117] "RemoveContainer" containerID="88feb8fabc24bac7bb24002313b8c9241562bd8c20cf4744bb0ef0da26a51d11"
Jul 16 22:19:16.398310 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:16.398284    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"]
Jul 16 22:19:16.406285 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:16.406258    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-39074f9f-85ff568bdb-dq8ww"]
Jul 16 22:19:18.125989 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:18.125942    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" path="/var/lib/kubelet/pods/7d0eaded-76c9-47cc-bdbf-18fd7972a1ce/volumes"
Jul 16 22:19:23.399902 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:23.399867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:23.612299 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:23.612245    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:23.839813 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:23.839766    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:24.058689 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:24.058604    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:25.076999 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.076963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:25.151466 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151433    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"]
Jul 16 22:19:25.151830 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151818    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.151876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151833    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.151876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151843    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.151876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151848    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.151876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151853    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.151876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151859    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.151876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151867    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:19:25.151876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151871    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:19:25.152102 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151959    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.152102 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151966    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.152102 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151972    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.152102 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151978    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:19:25.152102 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151984    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:19:25.152102 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.151990    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:19:25.155111 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.155095    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.157460 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.157437    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-23b64857-serving-cert\""
Jul 16 22:19:25.183501 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.183471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"]
Jul 16 22:19:25.251139 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.251100    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:25.254291 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.254264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzgfl\" (UniqueName: \"kubernetes.io/projected/77725de7-b73f-402b-88b1-bb1d59e67215-kube-api-access-nzgfl\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.254383 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.254355    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77725de7-b73f-402b-88b1-bb1d59e67215-openshift-service-ca\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.254446 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.254415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.355726 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.355624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzgfl\" (UniqueName: \"kubernetes.io/projected/77725de7-b73f-402b-88b1-bb1d59e67215-kube-api-access-nzgfl\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.355726 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.355685    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77725de7-b73f-402b-88b1-bb1d59e67215-openshift-service-ca\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.356023 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.355733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.356023 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:25.355837    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-serving-cert: secret "maas-api-e2e-ait-23b64857-serving-cert" not found
Jul 16 22:19:25.356023 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:25.355904    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls podName:77725de7-b73f-402b-88b1-bb1d59e67215 nodeName:}" failed. No retries permitted until 2026-07-16 22:19:25.855885085 +0000 UTC m=+1832.245158853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls") pod "maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" (UID: "77725de7-b73f-402b-88b1-bb1d59e67215") : secret "maas-api-e2e-ait-23b64857-serving-cert" not found
Jul 16 22:19:25.356440 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.356418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77725de7-b73f-402b-88b1-bb1d59e67215-openshift-service-ca\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.368319 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.368283    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzgfl\" (UniqueName: \"kubernetes.io/projected/77725de7-b73f-402b-88b1-bb1d59e67215-kube-api-access-nzgfl\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.416624 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.416585    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:25.785222 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.785165    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:25.860656 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.860617    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:25.863310 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:25.863288    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls\") pod \"maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:26.065122 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:26.065021    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:26.243123 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:26.242981    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"]
Jul 16 22:19:26.246016 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:19:26.245982    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77725de7_b73f_402b_88b1_bb1d59e67215.slice/crio-81ab037e19fb00eeb7663280c6b1671e680c789e67b4b66c77e1bcc4ee66c81c WatchSource:0}: Error finding container 81ab037e19fb00eeb7663280c6b1671e680c789e67b4b66c77e1bcc4ee66c81c: Status 404 returned error can't find the container with id 81ab037e19fb00eeb7663280c6b1671e680c789e67b4b66c77e1bcc4ee66c81c
Jul 16 22:19:26.437577 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:26.437390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerStarted","Data":"81ab037e19fb00eeb7663280c6b1671e680c789e67b4b66c77e1bcc4ee66c81c"}
Jul 16 22:19:26.466497 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:26.466467    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:26.585573 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:26.585539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:27.120763 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:27.120728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:27.443473 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:27.443442    2564 generic.go:356] "Generic (PLEG): container finished" podID="77725de7-b73f-402b-88b1-bb1d59e67215" containerID="a2ec0f5e3540d573b90b63294461e4e4a6f6049df7b6b2dfd257a406f5fcec76" exitCode=1
Jul 16 22:19:27.443944 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:27.443527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"a2ec0f5e3540d573b90b63294461e4e4a6f6049df7b6b2dfd257a406f5fcec76"}
Jul 16 22:19:27.443944 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:27.443768    2564 scope.go:117] "RemoveContainer" containerID="a2ec0f5e3540d573b90b63294461e4e4a6f6049df7b6b2dfd257a406f5fcec76"
Jul 16 22:19:29.456627 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:29.456597    2564 generic.go:356] "Generic (PLEG): container finished" podID="77725de7-b73f-402b-88b1-bb1d59e67215" containerID="2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05" exitCode=1
Jul 16 22:19:29.457073 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:29.456685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05"}
Jul 16 22:19:29.457073 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:29.456737    2564 scope.go:117] "RemoveContainer" containerID="a2ec0f5e3540d573b90b63294461e4e4a6f6049df7b6b2dfd257a406f5fcec76"
Jul 16 22:19:29.457073 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:29.456991    2564 scope.go:117] "RemoveContainer" containerID="2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05"
Jul 16 22:19:29.457204 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:29.457185    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:19:30.464156 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:30.464127    2564 scope.go:117] "RemoveContainer" containerID="2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05"
Jul 16 22:19:30.464548 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:30.464296    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:19:31.065842 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:31.065806    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:31.468989 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:31.468958    2564 scope.go:117] "RemoveContainer" containerID="2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05"
Jul 16 22:19:31.469429 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:31.469223    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:19:36.065642 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:36.065607    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:36.066152 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:36.066039    2564 scope.go:117] "RemoveContainer" containerID="2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05"
Jul 16 22:19:36.066235 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:36.066215    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:19:49.775581 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:49.775542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:50.119994 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:50.119894    2564 scope.go:117] "RemoveContainer" containerID="2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05"
Jul 16 22:19:50.120221 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:50.120208    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:19:52.579613 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:52.579578    2564 generic.go:356] "Generic (PLEG): container finished" podID="77725de7-b73f-402b-88b1-bb1d59e67215" containerID="49a538ba646d71f276138e0f143ebb330806a9677d48a3ec05d10ae0f3faf8b6" exitCode=1
Jul 16 22:19:52.580037 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:52.579638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"49a538ba646d71f276138e0f143ebb330806a9677d48a3ec05d10ae0f3faf8b6"}
Jul 16 22:19:52.580037 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:52.579688    2564 scope.go:117] "RemoveContainer" containerID="2c07d2405414c7c3f4e6867d115867fda5d943a1ef7a76811a447e6cc65bed05"
Jul 16 22:19:52.580037 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:52.580029    2564 scope.go:117] "RemoveContainer" containerID="49a538ba646d71f276138e0f143ebb330806a9677d48a3ec05d10ae0f3faf8b6"
Jul 16 22:19:52.580284 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:52.580261    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:19:56.065259 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:56.065223    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:56.065696 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:56.065286    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:19:56.065696 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:19:56.065595    2564 scope.go:117] "RemoveContainer" containerID="49a538ba646d71f276138e0f143ebb330806a9677d48a3ec05d10ae0f3faf8b6"
Jul 16 22:19:56.065797 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:19:56.065761    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:20:08.119800 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:08.119765    2564 scope.go:117] "RemoveContainer" containerID="49a538ba646d71f276138e0f143ebb330806a9677d48a3ec05d10ae0f3faf8b6"
Jul 16 22:20:08.120315 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:20:08.120036    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:20:20.120230 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:20.119997    2564 scope.go:117] "RemoveContainer" containerID="49a538ba646d71f276138e0f143ebb330806a9677d48a3ec05d10ae0f3faf8b6"
Jul 16 22:20:21.726983 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:21.726949    2564 generic.go:356] "Generic (PLEG): container finished" podID="77725de7-b73f-402b-88b1-bb1d59e67215" containerID="c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482" exitCode=1
Jul 16 22:20:21.726983 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:21.726989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482"}
Jul 16 22:20:21.727467 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:21.727039    2564 scope.go:117] "RemoveContainer" containerID="49a538ba646d71f276138e0f143ebb330806a9677d48a3ec05d10ae0f3faf8b6"
Jul 16 22:20:21.727467 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:21.727340    2564 scope.go:117] "RemoveContainer" containerID="c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482"
Jul 16 22:20:21.727545 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:20:21.727519    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:20:26.065304 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:26.065268    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:20:26.065713 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:26.065352    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:20:26.065713 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:26.065655    2564 scope.go:117] "RemoveContainer" containerID="c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482"
Jul 16 22:20:26.065833 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:20:26.065812    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:20:41.120194 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:41.120153    2564 scope.go:117] "RemoveContainer" containerID="c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482"
Jul 16 22:20:41.120672 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:20:41.120337    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:20:56.119197 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:20:56.119165    2564 scope.go:117] "RemoveContainer" containerID="c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482"
Jul 16 22:20:56.119584 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:20:56.119336    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:21:11.119719 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:11.119687    2564 scope.go:117] "RemoveContainer" containerID="c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482"
Jul 16 22:21:11.978171 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:11.978139    2564 generic.go:356] "Generic (PLEG): container finished" podID="77725de7-b73f-402b-88b1-bb1d59e67215" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b" exitCode=1
Jul 16 22:21:11.978370 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:11.978216    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"}
Jul 16 22:21:11.978370 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:11.978285    2564 scope.go:117] "RemoveContainer" containerID="c5a056a6c6cc3e15a437abe07df1b00b74cff56463fd02f5571bb1f8b9607482"
Jul 16 22:21:11.978600 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:11.978584    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:21:11.978788 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:21:11.978765    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:21:16.065318 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:16.065270    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:21:16.065733 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:16.065371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:21:16.065733 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:16.065685    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:21:16.065880 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:21:16.065859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:21:31.119752 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:31.119723    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:21:31.122242 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:21:31.119890    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:21:44.125868 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:44.125838    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:21:44.126493 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:21:44.126098    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:21:59.119322 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:21:59.119250    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:21:59.119694 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:21:59.119443    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:22:12.120020 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:12.119985    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:22:12.120512 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:22:12.120186    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:22:13.774487 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:13.774448    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:13.829995 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:13.829958    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:25.119267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:25.119236    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:22:25.119696 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:22:25.119401    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:22:38.119552 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:38.119520    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:22:39.417818 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:39.417786    2564 generic.go:356] "Generic (PLEG): container finished" podID="77725de7-b73f-402b-88b1-bb1d59e67215" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14" exitCode=1
Jul 16 22:22:39.418318 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:39.417859    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"}
Jul 16 22:22:39.418318 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:39.417967    2564 scope.go:117] "RemoveContainer" containerID="58aa69e4c14f2e50c3edc8e3b5ce5291ce99b064f6bf310b874bcc529271c81b"
Jul 16 22:22:39.418318 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:39.418299    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:22:39.418561 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:22:39.418542    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:22:41.065777 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:41.065743    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:22:41.066215 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:41.066110    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:22:41.066316 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:22:41.066297    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:22:41.452159 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:41.452129    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:41.552642 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:41.552597    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:46.065835 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:46.065802    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:22:46.066335 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:46.066319    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:22:46.066509 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:22:46.066490    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:22:51.025230 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:51.025183    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:51.123332 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:51.123292    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:58.119710 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:22:58.119669    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:22:58.120365 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:22:58.119854    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:23:13.120191 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:13.120160    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:23:13.120602 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:23:13.120341    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:23:25.120066 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:25.120031    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:23:25.120483 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:23:25.120233    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:23:36.119435 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:36.119406    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:23:36.119962 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:23:36.119595    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:23:50.119966 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:50.119927    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:23:50.120351 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:23:50.120111    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:23:54.196540 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:54.196509    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:23:54.202424 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:54.202391    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:23:54.203273 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:54.203252    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:23:54.209081 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:23:54.209063    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:24:03.121285 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:24:03.121258    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:24:03.121735 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:24:03.121484    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:24:15.119640 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:24:15.119602    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:24:15.120228 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:24:15.119827    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:24:30.120024 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:24:30.119997    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:24:30.120414 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:24:30.120152    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:24:45.120162 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:24:45.120067    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:24:45.120544 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:24:45.120276    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:24:59.119879 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:24:59.119848    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:24:59.120356 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:24:59.120078    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:25:12.119637 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:12.119596    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:25:12.120200 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:25:12.119813    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:25:16.130798 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:16.130765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:18.227040 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:18.227009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:26.119920 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:26.119889    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:25:26.120302 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:26.120218    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:25:27.268146 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:27.268059    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerStarted","Data":"56ed5ea2949363f0a7d0c22350b4af2e8ccb377ad1c44f2f2cf589246e14216c"}
Jul 16 22:25:27.268517 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:27.268280    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:25:27.288009 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:27.287961    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podStartSLOduration=356.985486295 podStartE2EDuration="6m2.287945865s" totalImagesPullingTime="5.30245957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:19:25 +0000 UTC" imagePullSessionsCount=7 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:25:27.285200405 +0000 UTC m=+2193.674474192" watchObservedRunningTime="2026-07-16 22:25:27.287945865 +0000 UTC m=+2193.677219649"
Jul 16 22:25:28.274332 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:28.274302    2564 generic.go:356] "Generic (PLEG): container finished" podID="77725de7-b73f-402b-88b1-bb1d59e67215" containerID="56ed5ea2949363f0a7d0c22350b4af2e8ccb377ad1c44f2f2cf589246e14216c" exitCode=1
Jul 16 22:25:28.274743 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:28.274399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"56ed5ea2949363f0a7d0c22350b4af2e8ccb377ad1c44f2f2cf589246e14216c"}
Jul 16 22:25:28.274743 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:28.274457    2564 scope.go:117] "RemoveContainer" containerID="52f699b03dbeeaf86a8886327b907cc78b5d6a6ed31687d05152ef7d73f8ec14"
Jul 16 22:25:28.274743 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:28.274591    2564 scope.go:117] "RemoveContainer" containerID="56ed5ea2949363f0a7d0c22350b4af2e8ccb377ad1c44f2f2cf589246e14216c"
Jul 16 22:25:28.274877 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:25:28.274776    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:25:29.281440 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:29.281411    2564 scope.go:117] "RemoveContainer" containerID="56ed5ea2949363f0a7d0c22350b4af2e8ccb377ad1c44f2f2cf589246e14216c"
Jul 16 22:25:29.281840 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:25:29.281581    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9_odh-ai-gateway-infra(77725de7-b73f-402b-88b1-bb1d59e67215)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" podUID="77725de7-b73f-402b-88b1-bb1d59e67215"
Jul 16 22:25:30.572706 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:30.572668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:30.673757 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:30.673725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:35.665696 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.665655    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"]
Jul 16 22:25:35.702704 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.702652    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:35.812236 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.812210    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:25:35.891659 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.891583    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzgfl\" (UniqueName: \"kubernetes.io/projected/77725de7-b73f-402b-88b1-bb1d59e67215-kube-api-access-nzgfl\") pod \"77725de7-b73f-402b-88b1-bb1d59e67215\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") "
Jul 16 22:25:35.891820 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.891668    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls\") pod \"77725de7-b73f-402b-88b1-bb1d59e67215\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") "
Jul 16 22:25:35.891820 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.891706    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77725de7-b73f-402b-88b1-bb1d59e67215-openshift-service-ca\") pod \"77725de7-b73f-402b-88b1-bb1d59e67215\" (UID: \"77725de7-b73f-402b-88b1-bb1d59e67215\") "
Jul 16 22:25:35.892208 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.892187    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77725de7-b73f-402b-88b1-bb1d59e67215-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "77725de7-b73f-402b-88b1-bb1d59e67215" (UID: "77725de7-b73f-402b-88b1-bb1d59e67215"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:25:35.893935 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.893895    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "77725de7-b73f-402b-88b1-bb1d59e67215" (UID: "77725de7-b73f-402b-88b1-bb1d59e67215"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:25:35.894133 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.894110    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77725de7-b73f-402b-88b1-bb1d59e67215-kube-api-access-nzgfl" (OuterVolumeSpecName: "kube-api-access-nzgfl") pod "77725de7-b73f-402b-88b1-bb1d59e67215" (UID: "77725de7-b73f-402b-88b1-bb1d59e67215"). InnerVolumeSpecName "kube-api-access-nzgfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:25:35.993226 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.993195    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzgfl\" (UniqueName: \"kubernetes.io/projected/77725de7-b73f-402b-88b1-bb1d59e67215-kube-api-access-nzgfl\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:25:35.993226 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.993222    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77725de7-b73f-402b-88b1-bb1d59e67215-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:25:35.993226 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:35.993232    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77725de7-b73f-402b-88b1-bb1d59e67215-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:25:36.317545 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:36.317511    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"
Jul 16 22:25:36.317722 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:36.317554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9" event={"ID":"77725de7-b73f-402b-88b1-bb1d59e67215","Type":"ContainerDied","Data":"81ab037e19fb00eeb7663280c6b1671e680c789e67b4b66c77e1bcc4ee66c81c"}
Jul 16 22:25:36.317722 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:36.317615    2564 scope.go:117] "RemoveContainer" containerID="56ed5ea2949363f0a7d0c22350b4af2e8ccb377ad1c44f2f2cf589246e14216c"
Jul 16 22:25:36.345550 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:36.345523    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"]
Jul 16 22:25:36.350077 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:36.350049    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-23b64857-ff86d8f8d-zfqh9"]
Jul 16 22:25:36.828474 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:36.828443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:38.126598 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:38.126560    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" path="/var/lib/kubelet/pods/77725de7-b73f-402b-88b1-bb1d59e67215/volumes"
Jul 16 22:25:42.699676 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:42.699618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:42.787734 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:42.787707    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:43.036147 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:43.036108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:44.227052 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.227018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:44.330354 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330316    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"]
Jul 16 22:25:44.330723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330710    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330724    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330734    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330740    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330748    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330754    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330762    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330767    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330772    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330777    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330784    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330791    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d0eaded-76c9-47cc-bdbf-18fd7972a1ce" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330798    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:25:44.330857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330803    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3c8fc2-0f00-48a8-bea1-680e679dfc8e" containerName="maas-api"
Jul 16 22:25:44.331267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330869    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.331267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330876    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.331267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330882    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.331267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330887    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.331267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330893    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.331267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.330900    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:25:44.335186 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.335155    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.339746 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.339718    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f757c4eb-serving-cert\""
Jul 16 22:25:44.357827 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.357792    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"]
Jul 16 22:25:44.358151 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.358118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:44.454675 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.454617    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:44.462175 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.462147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a626b6-68a7-414e-bb88-ee78f8428a31-openshift-service-ca\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.462298 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.462216    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29a626b6-68a7-414e-bb88-ee78f8428a31-maas-api-tls\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.462298 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.462242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5b8m\" (UniqueName: \"kubernetes.io/projected/29a626b6-68a7-414e-bb88-ee78f8428a31-kube-api-access-c5b8m\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.551064 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.550971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:44.563173 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.563146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29a626b6-68a7-414e-bb88-ee78f8428a31-maas-api-tls\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.563317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.563178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5b8m\" (UniqueName: \"kubernetes.io/projected/29a626b6-68a7-414e-bb88-ee78f8428a31-kube-api-access-c5b8m\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.563317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.563239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a626b6-68a7-414e-bb88-ee78f8428a31-openshift-service-ca\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.563820 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.563799    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a626b6-68a7-414e-bb88-ee78f8428a31-openshift-service-ca\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.565832 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.565812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29a626b6-68a7-414e-bb88-ee78f8428a31-maas-api-tls\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.579502 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.579482    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5b8m\" (UniqueName: \"kubernetes.io/projected/29a626b6-68a7-414e-bb88-ee78f8428a31-kube-api-access-c5b8m\") pod \"maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.646212 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.646186    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:44.809124 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:44.809102    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"]
Jul 16 22:25:44.811359 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:25:44.811330    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a626b6_68a7_414e_bb88_ee78f8428a31.slice/crio-7afbc0fb0545c31f80d4bee141d9c43db709ca4c4457fe409fac8ffecdab0636 WatchSource:0}: Error finding container 7afbc0fb0545c31f80d4bee141d9c43db709ca4c4457fe409fac8ffecdab0636: Status 404 returned error can't find the container with id 7afbc0fb0545c31f80d4bee141d9c43db709ca4c4457fe409fac8ffecdab0636
Jul 16 22:25:45.207181 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:45.207152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:45.309033 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:45.308999    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:45.368043 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:45.368006    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerStarted","Data":"5950ccef08b71e264c1f907c0b0ce00683bfe05dfd6721ff2ba6ad4788403ef8"}
Jul 16 22:25:45.368247 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:45.368225    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:45.368317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:45.368270    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerStarted","Data":"7afbc0fb0545c31f80d4bee141d9c43db709ca4c4457fe409fac8ffecdab0636"}
Jul 16 22:25:45.392424 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:45.392375    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podStartSLOduration=1.010163223 podStartE2EDuration="1.392361494s" totalImagesPullingTime="382.198271ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:25:45.39195309 +0000 UTC m=+2211.781226873" watchObservedRunningTime="2026-07-16 22:25:45.392361494 +0000 UTC m=+2211.781635312"
Jul 16 22:25:46.374295 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:46.374263    2564 generic.go:356] "Generic (PLEG): container finished" podID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerID="5950ccef08b71e264c1f907c0b0ce00683bfe05dfd6721ff2ba6ad4788403ef8" exitCode=1
Jul 16 22:25:46.374739 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:46.374356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerDied","Data":"5950ccef08b71e264c1f907c0b0ce00683bfe05dfd6721ff2ba6ad4788403ef8"}
Jul 16 22:25:46.374739 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:46.374585    2564 scope.go:117] "RemoveContainer" containerID="5950ccef08b71e264c1f907c0b0ce00683bfe05dfd6721ff2ba6ad4788403ef8"
Jul 16 22:25:47.383112 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:47.383074    2564 generic.go:356] "Generic (PLEG): container finished" podID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerID="72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c" exitCode=1
Jul 16 22:25:47.383502 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:47.383149    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerDied","Data":"72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c"}
Jul 16 22:25:47.383502 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:47.383205    2564 scope.go:117] "RemoveContainer" containerID="5950ccef08b71e264c1f907c0b0ce00683bfe05dfd6721ff2ba6ad4788403ef8"
Jul 16 22:25:47.383603 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:47.383506    2564 scope.go:117] "RemoveContainer" containerID="72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c"
Jul 16 22:25:47.383709 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:25:47.383687    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:25:48.389403 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:48.389371    2564 scope.go:117] "RemoveContainer" containerID="72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c"
Jul 16 22:25:48.389791 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:25:48.389537    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:25:50.368794 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:50.368758    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:50.369288 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:50.369180    2564 scope.go:117] "RemoveContainer" containerID="72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c"
Jul 16 22:25:50.369371 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:25:50.369351    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:25:51.889846 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:51.889810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:51.991264 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:51.991217    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:54.646389 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:54.646361    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:25:54.646865 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:54.646750    2564 scope.go:117] "RemoveContainer" containerID="72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c"
Jul 16 22:25:54.646977 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:25:54.646957    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:25:55.609552 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:25:55.609511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:07.119148 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:07.119118    2564 scope.go:117] "RemoveContainer" containerID="72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c"
Jul 16 22:26:08.495554 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:08.495522    2564 generic.go:356] "Generic (PLEG): container finished" podID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerID="6df8976323510ea66c3c90d00378f2ad920153062dfe793942a079b59cbfede4" exitCode=1
Jul 16 22:26:08.495983 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:08.495594    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerDied","Data":"6df8976323510ea66c3c90d00378f2ad920153062dfe793942a079b59cbfede4"}
Jul 16 22:26:08.495983 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:08.495660    2564 scope.go:117] "RemoveContainer" containerID="72406c3d683197a55cf0359d64472b451f89ec9828cf49e833477b797690318c"
Jul 16 22:26:08.496065 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:08.496039    2564 scope.go:117] "RemoveContainer" containerID="6df8976323510ea66c3c90d00378f2ad920153062dfe793942a079b59cbfede4"
Jul 16 22:26:08.496250 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:08.496231    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:26:10.369006 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:10.368972    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:26:10.369391 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:10.369315    2564 scope.go:117] "RemoveContainer" containerID="6df8976323510ea66c3c90d00378f2ad920153062dfe793942a079b59cbfede4"
Jul 16 22:26:10.369500 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:10.369479    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:26:14.646544 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:14.646455    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:26:14.646964 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:14.646832    2564 scope.go:117] "RemoveContainer" containerID="6df8976323510ea66c3c90d00378f2ad920153062dfe793942a079b59cbfede4"
Jul 16 22:26:14.647046 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:14.647026    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:26:28.119726 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:28.119690    2564 scope.go:117] "RemoveContainer" containerID="6df8976323510ea66c3c90d00378f2ad920153062dfe793942a079b59cbfede4"
Jul 16 22:26:29.602834 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:29.602801    2564 generic.go:356] "Generic (PLEG): container finished" podID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e" exitCode=1
Jul 16 22:26:29.603238 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:29.602884    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerDied","Data":"79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"}
Jul 16 22:26:29.603238 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:29.603000    2564 scope.go:117] "RemoveContainer" containerID="6df8976323510ea66c3c90d00378f2ad920153062dfe793942a079b59cbfede4"
Jul 16 22:26:29.603412 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:29.603396    2564 scope.go:117] "RemoveContainer" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"
Jul 16 22:26:29.603636 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:29.603611    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:26:30.368997 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:30.368962    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:26:30.609667 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:30.609636    2564 scope.go:117] "RemoveContainer" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"
Jul 16 22:26:30.610115 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:30.609894    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:26:34.646582 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:34.646550    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:26:34.649134 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:34.646949    2564 scope.go:117] "RemoveContainer" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"
Jul 16 22:26:34.649134 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:34.647140    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:26:48.119382 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:48.119345    2564 scope.go:117] "RemoveContainer" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"
Jul 16 22:26:48.119856 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:48.119557    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:26:59.119501 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:26:59.119473    2564 scope.go:117] "RemoveContainer" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"
Jul 16 22:26:59.120033 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:26:59.119654    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:27:13.119523 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:13.119493    2564 scope.go:117] "RemoveContainer" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"
Jul 16 22:27:13.828113 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:13.828081    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerStarted","Data":"591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"}
Jul 16 22:27:13.828315 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:13.828296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:27:14.835289 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:14.835257    2564 generic.go:356] "Generic (PLEG): container finished" podID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71" exitCode=1
Jul 16 22:27:14.835723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:14.835344    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerDied","Data":"591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"}
Jul 16 22:27:14.835723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:14.835402    2564 scope.go:117] "RemoveContainer" containerID="79d6b3a527fd22cf4446fdf137d10fe76e27621ae8b66a732b4562bfae21256e"
Jul 16 22:27:14.835723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:14.835552    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:27:14.835854 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:27:14.835748    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:27:15.843016 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:15.842988    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:27:15.843386 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:27:15.843173    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:27:24.647262 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:24.647219    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:27:24.647666 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:24.647625    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:27:24.647832 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:27:24.647811    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:27:37.119978 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:37.119948    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:27:37.120499 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:27:37.120124    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:27:42.328424 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:42.328386    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:42.418428 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:42.418391    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:48.356947 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:48.356787    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:48.675995 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:48.675891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:48.778067 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:48.778032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:51.119851 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:27:51.119815    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:27:51.120273 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:27:51.120001    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:28:04.121647 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:04.121610    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:28:04.122065 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:28:04.121787    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:28:17.119190 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:17.119159    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:28:17.119699 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:28:17.119329    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:28:28.120038 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:28.120004    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:28:28.120428 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:28:28.120171    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:28:40.119550 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:40.119520    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:28:41.273645 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:41.273611    2564 generic.go:356] "Generic (PLEG): container finished" podID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerID="fc89a0ebcee9d07a0ff80a745650ab3e96c11fb91eae6eb9a260f68464843d89" exitCode=1
Jul 16 22:28:41.273645 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:41.273650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerDied","Data":"fc89a0ebcee9d07a0ff80a745650ab3e96c11fb91eae6eb9a260f68464843d89"}
Jul 16 22:28:41.274121 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:41.273707    2564 scope.go:117] "RemoveContainer" containerID="591ec6d3c96dde852dd1cfc28e1bbbe2a0fc39122d821b64a8b4221cae6bed71"
Jul 16 22:28:41.274121 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:41.274037    2564 scope.go:117] "RemoveContainer" containerID="fc89a0ebcee9d07a0ff80a745650ab3e96c11fb91eae6eb9a260f68464843d89"
Jul 16 22:28:41.274246 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:28:41.274227    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:28:43.829190 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:43.829153    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:28:43.829567 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:43.829539    2564 scope.go:117] "RemoveContainer" containerID="fc89a0ebcee9d07a0ff80a745650ab3e96c11fb91eae6eb9a260f68464843d89"
Jul 16 22:28:43.829725 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:28:43.829703    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:28:44.647453 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:44.647419    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:28:44.647757 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:44.647744    2564 scope.go:117] "RemoveContainer" containerID="fc89a0ebcee9d07a0ff80a745650ab3e96c11fb91eae6eb9a260f68464843d89"
Jul 16 22:28:44.647960 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:28:44.647939    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg_odh-ai-gateway-infra(29a626b6-68a7-414e-bb88-ee78f8428a31)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31"
Jul 16 22:28:48.749558 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:48.749523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:48.815323 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:48.815294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:54.102954 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.102882    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"]
Jul 16 22:28:54.132568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.132510    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:54.233663 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.233636    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:28:54.240551 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.240528    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:28:54.240712 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.240697    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:28:54.246888 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.246872    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:28:54.255589 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.255568    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:28:54.304791 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.304762    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a626b6-68a7-414e-bb88-ee78f8428a31-openshift-service-ca\") pod \"29a626b6-68a7-414e-bb88-ee78f8428a31\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") "
Jul 16 22:28:54.304947 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.304826    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5b8m\" (UniqueName: \"kubernetes.io/projected/29a626b6-68a7-414e-bb88-ee78f8428a31-kube-api-access-c5b8m\") pod \"29a626b6-68a7-414e-bb88-ee78f8428a31\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") "
Jul 16 22:28:54.305001 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.304969    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29a626b6-68a7-414e-bb88-ee78f8428a31-maas-api-tls\") pod \"29a626b6-68a7-414e-bb88-ee78f8428a31\" (UID: \"29a626b6-68a7-414e-bb88-ee78f8428a31\") "
Jul 16 22:28:54.305159 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.305135    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29a626b6-68a7-414e-bb88-ee78f8428a31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29a626b6-68a7-414e-bb88-ee78f8428a31" (UID: "29a626b6-68a7-414e-bb88-ee78f8428a31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:28:54.305343 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.305321    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a626b6-68a7-414e-bb88-ee78f8428a31-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:28:54.307044 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.307021    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a626b6-68a7-414e-bb88-ee78f8428a31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29a626b6-68a7-414e-bb88-ee78f8428a31" (UID: "29a626b6-68a7-414e-bb88-ee78f8428a31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:28:54.307126 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.307086    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a626b6-68a7-414e-bb88-ee78f8428a31-kube-api-access-c5b8m" (OuterVolumeSpecName: "kube-api-access-c5b8m") pod "29a626b6-68a7-414e-bb88-ee78f8428a31" (UID: "29a626b6-68a7-414e-bb88-ee78f8428a31"). InnerVolumeSpecName "kube-api-access-c5b8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:28:54.348199 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.348155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg" event={"ID":"29a626b6-68a7-414e-bb88-ee78f8428a31","Type":"ContainerDied","Data":"7afbc0fb0545c31f80d4bee141d9c43db709ca4c4457fe409fac8ffecdab0636"}
Jul 16 22:28:54.348345 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.348279    2564 scope.go:117] "RemoveContainer" containerID="fc89a0ebcee9d07a0ff80a745650ab3e96c11fb91eae6eb9a260f68464843d89"
Jul 16 22:28:54.348490 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.348466    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"
Jul 16 22:28:54.372466 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.372436    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"]
Jul 16 22:28:54.378513 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.378489    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f757c4eb-547494c6bc-dfskg"]
Jul 16 22:28:54.405732 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.405714    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5b8m\" (UniqueName: \"kubernetes.io/projected/29a626b6-68a7-414e-bb88-ee78f8428a31-kube-api-access-c5b8m\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:28:54.405817 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.405734    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29a626b6-68a7-414e-bb88-ee78f8428a31-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:28:54.896336 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:54.896312    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:56.125996 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:28:56.125966    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" path="/var/lib/kubelet/pods/29a626b6-68a7-414e-bb88-ee78f8428a31/volumes"
Jul 16 22:29:00.638318 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:00.638287    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:00.726509 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:00.726481    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:01.230462 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:01.230428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:01.311282 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:01.311242    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:02.967213 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:02.967178    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:05.183733 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:05.183702    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:09.255847 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:09.255812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:09.361330 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:09.361285    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:09.459473 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:09.459443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:09.556153 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:09.556066    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:10.014730 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:10.014696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:28.760394 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760358    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"]
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760749    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760760    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760767    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760773    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760781    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760786    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760800    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760805    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760823    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760828    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760833    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760838    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760844    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760849    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760902    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760924    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760942    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760947    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760953    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:29:28.761087 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.760960    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77725de7-b73f-402b-88b1-bb1d59e67215" containerName="maas-api"
Jul 16 22:29:28.764263 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.764247    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"
Jul 16 22:29:28.767679 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.767663    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:29:28.773603 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.773579    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"]
Jul 16 22:29:28.898877 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:28.898841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e1e8f53-9bf8-47b2-890b-1f4b1e85774e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-d874m\" (UID: \"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"
Jul 16 22:29:29.000134 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:29.000104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e1e8f53-9bf8-47b2-890b-1f4b1e85774e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-d874m\" (UID: \"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"
Jul 16 22:29:29.000746 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:29.000720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e1e8f53-9bf8-47b2-890b-1f4b1e85774e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-d874m\" (UID: \"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"
Jul 16 22:29:29.074794 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:29.074698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"
Jul 16 22:29:29.422043 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:29.422020    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"]
Jul 16 22:29:29.424255 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:29:29.424223    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e1e8f53_9bf8_47b2_890b_1f4b1e85774e.slice/crio-3a65e844f291c7ab29f340737ef33284eaf6b5b0b3a41eea9e08c9cdfa0b27fb WatchSource:0}: Error finding container 3a65e844f291c7ab29f340737ef33284eaf6b5b0b3a41eea9e08c9cdfa0b27fb: Status 404 returned error can't find the container with id 3a65e844f291c7ab29f340737ef33284eaf6b5b0b3a41eea9e08c9cdfa0b27fb
Jul 16 22:29:29.526407 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:29.526375    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" event={"ID":"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e","Type":"ContainerStarted","Data":"3a65e844f291c7ab29f340737ef33284eaf6b5b0b3a41eea9e08c9cdfa0b27fb"}
Jul 16 22:29:30.532885 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:30.532849    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerID="318e8da5b9b9a5e0c78c407e6a2a7b725bdb66018e86d8978a14f592a427fbdf" exitCode=6
Jul 16 22:29:30.533341 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:30.532945    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" event={"ID":"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e","Type":"ContainerDied","Data":"318e8da5b9b9a5e0c78c407e6a2a7b725bdb66018e86d8978a14f592a427fbdf"}
Jul 16 22:29:30.533341 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:30.533119    2564 scope.go:117] "RemoveContainer" containerID="318e8da5b9b9a5e0c78c407e6a2a7b725bdb66018e86d8978a14f592a427fbdf"
Jul 16 22:29:31.541002 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:31.540971    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerID="e738816199d9bd29dea41f8f6082f01b6140e778eb28c98bd15e3749dd50cc26" exitCode=6
Jul 16 22:29:31.541417 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:31.541054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" event={"ID":"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e","Type":"ContainerDied","Data":"e738816199d9bd29dea41f8f6082f01b6140e778eb28c98bd15e3749dd50cc26"}
Jul 16 22:29:31.541417 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:31.541115    2564 scope.go:117] "RemoveContainer" containerID="318e8da5b9b9a5e0c78c407e6a2a7b725bdb66018e86d8978a14f592a427fbdf"
Jul 16 22:29:31.541417 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:31.541222    2564 scope.go:117] "RemoveContainer" containerID="e738816199d9bd29dea41f8f6082f01b6140e778eb28c98bd15e3749dd50cc26"
Jul 16 22:29:31.541548 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:29:31.541430    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-d874m_odh-ai-gateway-infra(1e1e8f53-9bf8-47b2-890b-1f4b1e85774e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e"
Jul 16 22:29:32.547748 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:32.547720    2564 scope.go:117] "RemoveContainer" containerID="e738816199d9bd29dea41f8f6082f01b6140e778eb28c98bd15e3749dd50cc26"
Jul 16 22:29:32.548173 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:29:32.547948    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-d874m_odh-ai-gateway-infra(1e1e8f53-9bf8-47b2-890b-1f4b1e85774e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e"
Jul 16 22:29:36.115716 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:36.115681    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:43.337986 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:43.337957    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:43.642315 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:43.642231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:43.892886 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:43.892797    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:47.119080 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:47.119047    2564 scope.go:117] "RemoveContainer" containerID="e738816199d9bd29dea41f8f6082f01b6140e778eb28c98bd15e3749dd50cc26"
Jul 16 22:29:47.621752 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:47.621707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" event={"ID":"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e","Type":"ContainerStarted","Data":"89c10e5c3aefc550b7d39923154c0b6a74a2e114b44c1c2210adc08d1d514ac0"}
Jul 16 22:29:47.668327 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:47.668274    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" podStartSLOduration=19.668258466 podStartE2EDuration="19.668258466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:29:47.667319022 +0000 UTC m=+2454.056592808" watchObservedRunningTime="2026-07-16 22:29:47.668258466 +0000 UTC m=+2454.057532252"
Jul 16 22:29:48.628394 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:48.628364    2564 generic.go:356] "Generic (PLEG): container finished" podID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerID="89c10e5c3aefc550b7d39923154c0b6a74a2e114b44c1c2210adc08d1d514ac0" exitCode=6
Jul 16 22:29:48.628856 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:48.628443    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" event={"ID":"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e","Type":"ContainerDied","Data":"89c10e5c3aefc550b7d39923154c0b6a74a2e114b44c1c2210adc08d1d514ac0"}
Jul 16 22:29:48.628856 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:48.628521    2564 scope.go:117] "RemoveContainer" containerID="e738816199d9bd29dea41f8f6082f01b6140e778eb28c98bd15e3749dd50cc26"
Jul 16 22:29:48.628856 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:48.628829    2564 scope.go:117] "RemoveContainer" containerID="89c10e5c3aefc550b7d39923154c0b6a74a2e114b44c1c2210adc08d1d514ac0"
Jul 16 22:29:48.629120 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:29:48.629099    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-d874m_odh-ai-gateway-infra(1e1e8f53-9bf8-47b2-890b-1f4b1e85774e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e"
Jul 16 22:29:48.685976 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:48.685943    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"]
Jul 16 22:29:49.794152 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:49.794124    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"
Jul 16 22:29:49.882896 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:49.882861    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e1e8f53-9bf8-47b2-890b-1f4b1e85774e-maas-api-service-ca\") pod \"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e\" (UID: \"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e\") "
Jul 16 22:29:49.883203 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:49.883180    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e1e8f53-9bf8-47b2-890b-1f4b1e85774e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" (UID: "1e1e8f53-9bf8-47b2-890b-1f4b1e85774e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:29:49.984500 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:49.984464    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e1e8f53-9bf8-47b2-890b-1f4b1e85774e-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:29:50.641201 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:50.641164    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m" event={"ID":"1e1e8f53-9bf8-47b2-890b-1f4b1e85774e","Type":"ContainerDied","Data":"3a65e844f291c7ab29f340737ef33284eaf6b5b0b3a41eea9e08c9cdfa0b27fb"}
Jul 16 22:29:50.641386 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:50.641206    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"
Jul 16 22:29:50.641386 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:50.641228    2564 scope.go:117] "RemoveContainer" containerID="89c10e5c3aefc550b7d39923154c0b6a74a2e114b44c1c2210adc08d1d514ac0"
Jul 16 22:29:50.671488 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:50.671460    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"]
Jul 16 22:29:50.678737 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:50.678715    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-d874m"]
Jul 16 22:29:52.125465 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:52.125433    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" path="/var/lib/kubelet/pods/1e1e8f53-9bf8-47b2-890b-1f4b1e85774e/volumes"
Jul 16 22:29:56.977611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:56.977569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:29:59.560664 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:29:59.560628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:00.145388 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145350    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt"]
Jul 16 22:30:00.145734 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145721    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.145779 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145735    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.145779 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145742    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:30:00.145779 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145748    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:30:00.145779 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145758    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.145779 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145764    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.145779 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145776    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.145998 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145781    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.145998 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145839    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.145998 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145846    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a626b6-68a7-414e-bb88-ee78f8428a31" containerName="maas-api"
Jul 16 22:30:00.145998 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.145856    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:00.148820 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.148799    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt"
Jul 16 22:30:00.152710 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.152690    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:30:00.158529 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.158509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt"
Jul 16 22:30:00.164084 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.164055    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt"]
Jul 16 22:30:00.317020 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.316985    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt"]
Jul 16 22:30:00.321119 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:30:00.321093    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e4d2da_63bf_4e79_a13e_c4706abc61b5.slice/crio-ecb8fab86e6fb807e13d297c568b3c874af2de7a8d3f9b7e03b0a3157de8eb99 WatchSource:0}: Error finding container ecb8fab86e6fb807e13d297c568b3c874af2de7a8d3f9b7e03b0a3157de8eb99: Status 404 returned error can't find the container with id ecb8fab86e6fb807e13d297c568b3c874af2de7a8d3f9b7e03b0a3157de8eb99
Jul 16 22:30:00.700886 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.700847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt" event={"ID":"36e4d2da-63bf-4e79-a13e-c4706abc61b5","Type":"ContainerStarted","Data":"dff200b4ab61cbe5b4120e3dc861ca7a26d7531e6027bb735c45be1d178830e5"}
Jul 16 22:30:00.701327 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.700900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt" event={"ID":"36e4d2da-63bf-4e79-a13e-c4706abc61b5","Type":"ContainerStarted","Data":"ecb8fab86e6fb807e13d297c568b3c874af2de7a8d3f9b7e03b0a3157de8eb99"}
Jul 16 22:30:00.727392 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:00.727349    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt" podStartSLOduration=0.727334854 podStartE2EDuration="727.334854ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:30:00.72460122 +0000 UTC m=+2467.113875007" watchObservedRunningTime="2026-07-16 22:30:00.727334854 +0000 UTC m=+2467.116608640"
Jul 16 22:30:01.707263 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:01.707226    2564 generic.go:356] "Generic (PLEG): container finished" podID="36e4d2da-63bf-4e79-a13e-c4706abc61b5" containerID="dff200b4ab61cbe5b4120e3dc861ca7a26d7531e6027bb735c45be1d178830e5" exitCode=0
Jul 16 22:30:01.707650 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:01.707280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt" event={"ID":"36e4d2da-63bf-4e79-a13e-c4706abc61b5","Type":"ContainerDied","Data":"dff200b4ab61cbe5b4120e3dc861ca7a26d7531e6027bb735c45be1d178830e5"}
Jul 16 22:30:02.850018 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:02.849995    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt"
Jul 16 22:30:03.718306 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:03.718275    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt"
Jul 16 22:30:03.718306 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:03.718297    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-g6grt" event={"ID":"36e4d2da-63bf-4e79-a13e-c4706abc61b5","Type":"ContainerDied","Data":"ecb8fab86e6fb807e13d297c568b3c874af2de7a8d3f9b7e03b0a3157de8eb99"}
Jul 16 22:30:03.718515 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:03.718366    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecb8fab86e6fb807e13d297c568b3c874af2de7a8d3f9b7e03b0a3157de8eb99"
Jul 16 22:30:05.659360 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:05.659324    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:05.762795 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:05.762765    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:05.854119 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:05.854080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:16.267931 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:16.267865    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:16.507350 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:16.507312    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:17.388190 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:17.388150    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:20.714576 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.714541    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"]
Jul 16 22:30:20.714982 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.714930    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e4d2da-63bf-4e79-a13e-c4706abc61b5" containerName="cleanup"
Jul 16 22:30:20.714982 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.714945    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e4d2da-63bf-4e79-a13e-c4706abc61b5" containerName="cleanup"
Jul 16 22:30:20.715056 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.715038    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e1e8f53-9bf8-47b2-890b-1f4b1e85774e" containerName="revoke-keys"
Jul 16 22:30:20.715056 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.715045    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e4d2da-63bf-4e79-a13e-c4706abc61b5" containerName="cleanup"
Jul 16 22:30:20.718088 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.718072    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"
Jul 16 22:30:20.720760 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.720737    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:30:20.726928 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.726890    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"]
Jul 16 22:30:20.855900 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.855860    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c35769d-541b-4aba-a4e5-576e0e979e98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-ccc7t\" (UID: \"6c35769d-541b-4aba-a4e5-576e0e979e98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"
Jul 16 22:30:20.957233 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.957197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c35769d-541b-4aba-a4e5-576e0e979e98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-ccc7t\" (UID: \"6c35769d-541b-4aba-a4e5-576e0e979e98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"
Jul 16 22:30:20.957805 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:20.957785    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c35769d-541b-4aba-a4e5-576e0e979e98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-ccc7t\" (UID: \"6c35769d-541b-4aba-a4e5-576e0e979e98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"
Jul 16 22:30:21.028720 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:21.028624    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"
Jul 16 22:30:21.168901 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:21.168878    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"]
Jul 16 22:30:21.171197 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:30:21.171174    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c35769d_541b_4aba_a4e5_576e0e979e98.slice/crio-53034b64e564450bf3f687e0430efdc1e9b5b6f90e73c9bf814b69b90ff51908 WatchSource:0}: Error finding container 53034b64e564450bf3f687e0430efdc1e9b5b6f90e73c9bf814b69b90ff51908: Status 404 returned error can't find the container with id 53034b64e564450bf3f687e0430efdc1e9b5b6f90e73c9bf814b69b90ff51908
Jul 16 22:30:21.808924 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:21.808885    2564 generic.go:356] "Generic (PLEG): container finished" podID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerID="4c25c7df2b8afe314a88c2fc5bd578c1256efdcfea388cce941ff205f7e76091" exitCode=6
Jul 16 22:30:21.809268 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:21.808957    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" event={"ID":"6c35769d-541b-4aba-a4e5-576e0e979e98","Type":"ContainerDied","Data":"4c25c7df2b8afe314a88c2fc5bd578c1256efdcfea388cce941ff205f7e76091"}
Jul 16 22:30:21.809268 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:21.809017    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" event={"ID":"6c35769d-541b-4aba-a4e5-576e0e979e98","Type":"ContainerStarted","Data":"53034b64e564450bf3f687e0430efdc1e9b5b6f90e73c9bf814b69b90ff51908"}
Jul 16 22:30:21.809268 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:21.809185    2564 scope.go:117] "RemoveContainer" containerID="4c25c7df2b8afe314a88c2fc5bd578c1256efdcfea388cce941ff205f7e76091"
Jul 16 22:30:22.815496 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:22.815401    2564 generic.go:356] "Generic (PLEG): container finished" podID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerID="fd5cd822d8130211a4cdb956bf4888787cfed3a1bd234b9b5059160e149003ea" exitCode=6
Jul 16 22:30:22.815496 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:22.815450    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" event={"ID":"6c35769d-541b-4aba-a4e5-576e0e979e98","Type":"ContainerDied","Data":"fd5cd822d8130211a4cdb956bf4888787cfed3a1bd234b9b5059160e149003ea"}
Jul 16 22:30:22.816018 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:22.815495    2564 scope.go:117] "RemoveContainer" containerID="4c25c7df2b8afe314a88c2fc5bd578c1256efdcfea388cce941ff205f7e76091"
Jul 16 22:30:22.816018 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:22.815705    2564 scope.go:117] "RemoveContainer" containerID="fd5cd822d8130211a4cdb956bf4888787cfed3a1bd234b9b5059160e149003ea"
Jul 16 22:30:22.816018 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:30:22.815984    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-ccc7t_odh-ai-gateway-infra(6c35769d-541b-4aba-a4e5-576e0e979e98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98"
Jul 16 22:30:23.822235 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:23.822202    2564 scope.go:117] "RemoveContainer" containerID="fd5cd822d8130211a4cdb956bf4888787cfed3a1bd234b9b5059160e149003ea"
Jul 16 22:30:23.822609 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:30:23.822410    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-ccc7t_odh-ai-gateway-infra(6c35769d-541b-4aba-a4e5-576e0e979e98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98"
Jul 16 22:30:23.880221 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:23.880191    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:25.614017 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:25.613979    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:27.240949 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:27.240889    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:35.117033 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:35.117001    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:35.276383 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:35.276340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:35.852778 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:35.852741    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:36.119461 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:36.119374    2564 scope.go:117] "RemoveContainer" containerID="fd5cd822d8130211a4cdb956bf4888787cfed3a1bd234b9b5059160e149003ea"
Jul 16 22:30:36.888996 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:36.888955    2564 generic.go:356] "Generic (PLEG): container finished" podID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerID="182d63ba39692a6b7251460be99fd17649d08459b47348e30d680b4bf734770e" exitCode=6
Jul 16 22:30:36.889148 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:36.889017    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" event={"ID":"6c35769d-541b-4aba-a4e5-576e0e979e98","Type":"ContainerDied","Data":"182d63ba39692a6b7251460be99fd17649d08459b47348e30d680b4bf734770e"}
Jul 16 22:30:36.889148 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:36.889080    2564 scope.go:117] "RemoveContainer" containerID="fd5cd822d8130211a4cdb956bf4888787cfed3a1bd234b9b5059160e149003ea"
Jul 16 22:30:36.889429 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:36.889404    2564 scope.go:117] "RemoveContainer" containerID="182d63ba39692a6b7251460be99fd17649d08459b47348e30d680b4bf734770e"
Jul 16 22:30:36.889716 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:30:36.889688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-ccc7t_odh-ai-gateway-infra(6c35769d-541b-4aba-a4e5-576e0e979e98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98"
Jul 16 22:30:37.922366 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:37.922325    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"]
Jul 16 22:30:38.065348 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.065322    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"
Jul 16 22:30:38.214420 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.214395    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c35769d-541b-4aba-a4e5-576e0e979e98-maas-api-service-ca\") pod \"6c35769d-541b-4aba-a4e5-576e0e979e98\" (UID: \"6c35769d-541b-4aba-a4e5-576e0e979e98\") "
Jul 16 22:30:38.214774 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.214754    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c35769d-541b-4aba-a4e5-576e0e979e98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6c35769d-541b-4aba-a4e5-576e0e979e98" (UID: "6c35769d-541b-4aba-a4e5-576e0e979e98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:30:38.315105 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.315067    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c35769d-541b-4aba-a4e5-576e0e979e98-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:30:38.901439 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.901407    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"
Jul 16 22:30:38.901643 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.901403    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t" event={"ID":"6c35769d-541b-4aba-a4e5-576e0e979e98","Type":"ContainerDied","Data":"53034b64e564450bf3f687e0430efdc1e9b5b6f90e73c9bf814b69b90ff51908"}
Jul 16 22:30:38.901643 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.901548    2564 scope.go:117] "RemoveContainer" containerID="182d63ba39692a6b7251460be99fd17649d08459b47348e30d680b4bf734770e"
Jul 16 22:30:38.923829 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.923806    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"]
Jul 16 22:30:38.928391 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:38.928369    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-ccc7t"]
Jul 16 22:30:40.125782 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:40.125744    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" path="/var/lib/kubelet/pods/6c35769d-541b-4aba-a4e5-576e0e979e98/volumes"
Jul 16 22:30:43.329524 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:43.329489    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:43.438439 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:43.438405    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:43.561734 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:43.561699    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:43.953444 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:43.953403    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:44.358442 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:44.358366    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:47.010172 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.010135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:47.200206 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.200177    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:47.292866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.292793    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"]
Jul 16 22:30:47.293203 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.293190    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:30:47.293291 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.293205    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:30:47.293291 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.293223    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:30:47.293291 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.293230    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:30:47.293386 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.293309    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:30:47.293386 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.293317    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:30:47.293386 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.293323    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:30:47.297727 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.297705    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.313845 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.308232    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5d6c742e-serving-cert\""
Jul 16 22:30:47.322303 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.322279    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"]
Jul 16 22:30:47.399098 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.399055    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b693d2a-2b40-4cb7-b55c-ff669d879123-openshift-service-ca\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.399294 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.399110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b4nx\" (UniqueName: \"kubernetes.io/projected/5b693d2a-2b40-4cb7-b55c-ff669d879123-kube-api-access-4b4nx\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.399294 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.399173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b693d2a-2b40-4cb7-b55c-ff669d879123-maas-api-tls\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.432875 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.432839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:47.499660 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.499628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b693d2a-2b40-4cb7-b55c-ff669d879123-openshift-service-ca\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.499811 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.499668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4nx\" (UniqueName: \"kubernetes.io/projected/5b693d2a-2b40-4cb7-b55c-ff669d879123-kube-api-access-4b4nx\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.499811 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.499691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b693d2a-2b40-4cb7-b55c-ff669d879123-maas-api-tls\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.500292 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.500266    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b693d2a-2b40-4cb7-b55c-ff669d879123-openshift-service-ca\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.502372 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.502352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b693d2a-2b40-4cb7-b55c-ff669d879123-maas-api-tls\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.529771 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.529745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b4nx\" (UniqueName: \"kubernetes.io/projected/5b693d2a-2b40-4cb7-b55c-ff669d879123-kube-api-access-4b4nx\") pod \"maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.607527 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.607456    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:47.766129 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.766103    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"]
Jul 16 22:30:47.768290 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:30:47.768260    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b693d2a_2b40_4cb7_b55c_ff669d879123.slice/crio-70459be49f6af7d290a2b73dd9236a3a25fe953b445d8a73fa21660f1b737549 WatchSource:0}: Error finding container 70459be49f6af7d290a2b73dd9236a3a25fe953b445d8a73fa21660f1b737549: Status 404 returned error can't find the container with id 70459be49f6af7d290a2b73dd9236a3a25fe953b445d8a73fa21660f1b737549
Jul 16 22:30:47.769665 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.769644    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:30:47.951654 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:47.951616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l" event={"ID":"5b693d2a-2b40-4cb7-b55c-ff669d879123","Type":"ContainerStarted","Data":"70459be49f6af7d290a2b73dd9236a3a25fe953b445d8a73fa21660f1b737549"}
Jul 16 22:30:48.440496 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:48.440461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:48.586202 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:48.586163    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:48.961327 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:48.961291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l" event={"ID":"5b693d2a-2b40-4cb7-b55c-ff669d879123","Type":"ContainerStarted","Data":"b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc"}
Jul 16 22:30:48.961489 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:48.961360    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:51.465177 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:51.465143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:52.244234 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:52.244197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:52.453263 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:52.453225    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:52.752763 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:52.752728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:54.972157 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:54.972127    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:30:54.992488 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:54.992443    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l" podStartSLOduration=6.928895186 podStartE2EDuration="7.992428395s" totalImagesPullingTime="1.063533209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:30:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:30:49.002209321 +0000 UTC m=+2515.391483130" watchObservedRunningTime="2026-07-16 22:30:54.992428395 +0000 UTC m=+2521.381702181"
Jul 16 22:30:58.730407 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:58.730360    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:58.802314 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:30:58.802275    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:09.265568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.265529    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"]
Jul 16 22:31:09.266068 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.265763    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l" podUID="5b693d2a-2b40-4cb7-b55c-ff669d879123" containerName="maas-api" containerID="cri-o://b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc" gracePeriod=30
Jul 16 22:31:09.306113 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.306062    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:09.518992 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.518904    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:31:09.602432 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.602400    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b693d2a-2b40-4cb7-b55c-ff669d879123-openshift-service-ca\") pod \"5b693d2a-2b40-4cb7-b55c-ff669d879123\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") "
Jul 16 22:31:09.602595 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.602497    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b693d2a-2b40-4cb7-b55c-ff669d879123-maas-api-tls\") pod \"5b693d2a-2b40-4cb7-b55c-ff669d879123\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") "
Jul 16 22:31:09.602595 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.602548    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b4nx\" (UniqueName: \"kubernetes.io/projected/5b693d2a-2b40-4cb7-b55c-ff669d879123-kube-api-access-4b4nx\") pod \"5b693d2a-2b40-4cb7-b55c-ff669d879123\" (UID: \"5b693d2a-2b40-4cb7-b55c-ff669d879123\") "
Jul 16 22:31:09.602764 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.602740    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b693d2a-2b40-4cb7-b55c-ff669d879123-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b693d2a-2b40-4cb7-b55c-ff669d879123" (UID: "5b693d2a-2b40-4cb7-b55c-ff669d879123"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:31:09.604873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.604842    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b693d2a-2b40-4cb7-b55c-ff669d879123-kube-api-access-4b4nx" (OuterVolumeSpecName: "kube-api-access-4b4nx") pod "5b693d2a-2b40-4cb7-b55c-ff669d879123" (UID: "5b693d2a-2b40-4cb7-b55c-ff669d879123"). InnerVolumeSpecName "kube-api-access-4b4nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:31:09.604873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.604841    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b693d2a-2b40-4cb7-b55c-ff669d879123-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b693d2a-2b40-4cb7-b55c-ff669d879123" (UID: "5b693d2a-2b40-4cb7-b55c-ff669d879123"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:31:09.704046 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.704005    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b693d2a-2b40-4cb7-b55c-ff669d879123-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:31:09.704046 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.704040    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b693d2a-2b40-4cb7-b55c-ff669d879123-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:31:09.704046 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:09.704049    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b4nx\" (UniqueName: \"kubernetes.io/projected/5b693d2a-2b40-4cb7-b55c-ff669d879123-kube-api-access-4b4nx\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:31:10.069929 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.069873    2564 generic.go:356] "Generic (PLEG): container finished" podID="5b693d2a-2b40-4cb7-b55c-ff669d879123" containerID="b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc" exitCode=0
Jul 16 22:31:10.069929 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.069927    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"
Jul 16 22:31:10.070137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.069959    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l" event={"ID":"5b693d2a-2b40-4cb7-b55c-ff669d879123","Type":"ContainerDied","Data":"b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc"}
Jul 16 22:31:10.070137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.070018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l" event={"ID":"5b693d2a-2b40-4cb7-b55c-ff669d879123","Type":"ContainerDied","Data":"70459be49f6af7d290a2b73dd9236a3a25fe953b445d8a73fa21660f1b737549"}
Jul 16 22:31:10.070137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.070048    2564 scope.go:117] "RemoveContainer" containerID="b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc"
Jul 16 22:31:10.079265 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.079246    2564 scope.go:117] "RemoveContainer" containerID="b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc"
Jul 16 22:31:10.079537 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:31:10.079519    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc\": container with ID starting with b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc not found: ID does not exist" containerID="b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc"
Jul 16 22:31:10.079583 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.079547    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc"} err="failed to get container status \"b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc\": rpc error: code = NotFound desc = could not find container \"b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc\": container with ID starting with b5344e2240d289cbb112f4b086af719c2c2b64b945ce7bfdce9f61705cc96dbc not found: ID does not exist"
Jul 16 22:31:10.092032 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.092006    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"]
Jul 16 22:31:10.096869 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.096846    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5d6c742e-8c7f64c74-wnp2l"]
Jul 16 22:31:10.126498 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.126460    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b693d2a-2b40-4cb7-b55c-ff669d879123" path="/var/lib/kubelet/pods/5b693d2a-2b40-4cb7-b55c-ff669d879123/volumes"
Jul 16 22:31:10.180545 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.180508    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"]
Jul 16 22:31:10.180943 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.180923    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:31:10.180993 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.180945    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c35769d-541b-4aba-a4e5-576e0e979e98" containerName="revoke-keys"
Jul 16 22:31:10.180993 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.180958    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b693d2a-2b40-4cb7-b55c-ff669d879123" containerName="maas-api"
Jul 16 22:31:10.180993 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.180963    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b693d2a-2b40-4cb7-b55c-ff669d879123" containerName="maas-api"
Jul 16 22:31:10.181090 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.181066    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b693d2a-2b40-4cb7-b55c-ff669d879123" containerName="maas-api"
Jul 16 22:31:10.185250 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.185236    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"
Jul 16 22:31:10.188723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.188704    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:31:10.193519 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.193491    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"]
Jul 16 22:31:10.310987 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.310944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58ffac2e-6759-40d8-9122-fe214d723f1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc\" (UID: \"58ffac2e-6759-40d8-9122-fe214d723f1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"
Jul 16 22:31:10.411580 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.411495    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58ffac2e-6759-40d8-9122-fe214d723f1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc\" (UID: \"58ffac2e-6759-40d8-9122-fe214d723f1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"
Jul 16 22:31:10.412188 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.412165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58ffac2e-6759-40d8-9122-fe214d723f1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc\" (UID: \"58ffac2e-6759-40d8-9122-fe214d723f1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"
Jul 16 22:31:10.496430 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.496394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"
Jul 16 22:31:10.635153 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:10.635126    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"]
Jul 16 22:31:10.638084 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:31:10.638060    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ffac2e_6759_40d8_9122_fe214d723f1c.slice/crio-91cb88abd24649ae7b3dff9a369779c813a8430bc0775c5c76f22666b1642a3f WatchSource:0}: Error finding container 91cb88abd24649ae7b3dff9a369779c813a8430bc0775c5c76f22666b1642a3f: Status 404 returned error can't find the container with id 91cb88abd24649ae7b3dff9a369779c813a8430bc0775c5c76f22666b1642a3f
Jul 16 22:31:11.077408 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:11.077368    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" event={"ID":"58ffac2e-6759-40d8-9122-fe214d723f1c","Type":"ContainerStarted","Data":"91011118ce02a878d08494229cec927b87d0b3b7fba5c2b6c87646acd722f0e8"}
Jul 16 22:31:11.077618 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:11.077434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" event={"ID":"58ffac2e-6759-40d8-9122-fe214d723f1c","Type":"ContainerStarted","Data":"91cb88abd24649ae7b3dff9a369779c813a8430bc0775c5c76f22666b1642a3f"}
Jul 16 22:31:11.094959 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:11.094885    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" podStartSLOduration=1.09487032 podStartE2EDuration="1.09487032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:31:11.093495893 +0000 UTC m=+2537.482769679" watchObservedRunningTime="2026-07-16 22:31:11.09487032 +0000 UTC m=+2537.484144105"
Jul 16 22:31:11.849712 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:11.849674    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:11.930167 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:11.930138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:12.084701 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:12.084665    2564 generic.go:356] "Generic (PLEG): container finished" podID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerID="91011118ce02a878d08494229cec927b87d0b3b7fba5c2b6c87646acd722f0e8" exitCode=6
Jul 16 22:31:12.084875 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:12.084754    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" event={"ID":"58ffac2e-6759-40d8-9122-fe214d723f1c","Type":"ContainerDied","Data":"91011118ce02a878d08494229cec927b87d0b3b7fba5c2b6c87646acd722f0e8"}
Jul 16 22:31:12.084961 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:12.084943    2564 scope.go:117] "RemoveContainer" containerID="91011118ce02a878d08494229cec927b87d0b3b7fba5c2b6c87646acd722f0e8"
Jul 16 22:31:13.092419 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:13.092331    2564 generic.go:356] "Generic (PLEG): container finished" podID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerID="d497cd3f7b4749792f9b219fd68a791b2979afea4c43f5ba95d948e3d664cd6a" exitCode=6
Jul 16 22:31:13.092823 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:13.092421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" event={"ID":"58ffac2e-6759-40d8-9122-fe214d723f1c","Type":"ContainerDied","Data":"d497cd3f7b4749792f9b219fd68a791b2979afea4c43f5ba95d948e3d664cd6a"}
Jul 16 22:31:13.092823 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:13.092483    2564 scope.go:117] "RemoveContainer" containerID="91011118ce02a878d08494229cec927b87d0b3b7fba5c2b6c87646acd722f0e8"
Jul 16 22:31:13.092823 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:13.092590    2564 scope.go:117] "RemoveContainer" containerID="d497cd3f7b4749792f9b219fd68a791b2979afea4c43f5ba95d948e3d664cd6a"
Jul 16 22:31:13.092978 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:31:13.092821    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc_odh-ai-gateway-infra(58ffac2e-6759-40d8-9122-fe214d723f1c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c"
Jul 16 22:31:14.100122 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:14.100088    2564 scope.go:117] "RemoveContainer" containerID="d497cd3f7b4749792f9b219fd68a791b2979afea4c43f5ba95d948e3d664cd6a"
Jul 16 22:31:14.100520 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:31:14.100301    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc_odh-ai-gateway-infra(58ffac2e-6759-40d8-9122-fe214d723f1c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c"
Jul 16 22:31:14.184610 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:14.184574    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:21.227594 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:21.227554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:21.430401 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:21.430372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:24.947940 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:24.947884    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:25.175331 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:25.175293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:25.317866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:25.317771    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:25.427611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:25.427577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:25.603248 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:25.603148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:29.119819 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:29.119790    2564 scope.go:117] "RemoveContainer" containerID="d497cd3f7b4749792f9b219fd68a791b2979afea4c43f5ba95d948e3d664cd6a"
Jul 16 22:31:30.178364 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:30.178276    2564 generic.go:356] "Generic (PLEG): container finished" podID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerID="8834c563c6a11e591dfdfc2f8ce739c2210f81dfe17c1c2a0d93b7d70421c117" exitCode=6
Jul 16 22:31:30.178364 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:30.178349    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" event={"ID":"58ffac2e-6759-40d8-9122-fe214d723f1c","Type":"ContainerDied","Data":"8834c563c6a11e591dfdfc2f8ce739c2210f81dfe17c1c2a0d93b7d70421c117"}
Jul 16 22:31:30.178950 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:30.178426    2564 scope.go:117] "RemoveContainer" containerID="d497cd3f7b4749792f9b219fd68a791b2979afea4c43f5ba95d948e3d664cd6a"
Jul 16 22:31:30.178950 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:30.178773    2564 scope.go:117] "RemoveContainer" containerID="8834c563c6a11e591dfdfc2f8ce739c2210f81dfe17c1c2a0d93b7d70421c117"
Jul 16 22:31:30.179050 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:31:30.178997    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc_odh-ai-gateway-infra(58ffac2e-6759-40d8-9122-fe214d723f1c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c"
Jul 16 22:31:31.243408 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:31.243373    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"]
Jul 16 22:31:31.380510 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:31.380487    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"
Jul 16 22:31:31.486991 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:31.486961    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58ffac2e-6759-40d8-9122-fe214d723f1c-maas-api-service-ca\") pod \"58ffac2e-6759-40d8-9122-fe214d723f1c\" (UID: \"58ffac2e-6759-40d8-9122-fe214d723f1c\") "
Jul 16 22:31:31.487352 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:31.487327    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58ffac2e-6759-40d8-9122-fe214d723f1c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "58ffac2e-6759-40d8-9122-fe214d723f1c" (UID: "58ffac2e-6759-40d8-9122-fe214d723f1c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:31:31.587624 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:31.587549    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58ffac2e-6759-40d8-9122-fe214d723f1c-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:31:32.192446 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:32.192419    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"
Jul 16 22:31:32.192612 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:32.192446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc" event={"ID":"58ffac2e-6759-40d8-9122-fe214d723f1c","Type":"ContainerDied","Data":"91cb88abd24649ae7b3dff9a369779c813a8430bc0775c5c76f22666b1642a3f"}
Jul 16 22:31:32.192612 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:32.192507    2564 scope.go:117] "RemoveContainer" containerID="8834c563c6a11e591dfdfc2f8ce739c2210f81dfe17c1c2a0d93b7d70421c117"
Jul 16 22:31:32.222492 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:32.222470    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"]
Jul 16 22:31:32.233498 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:32.233476    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d3f2b144-4p9nc"]
Jul 16 22:31:33.994821 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:33.994786    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:34.125883 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:34.125849    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" path="/var/lib/kubelet/pods/58ffac2e-6759-40d8-9122-fe214d723f1c/volumes"
Jul 16 22:31:34.248435 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:34.248332    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:34.412241 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:34.412188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:34.538117 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:34.538017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:41.620270 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:41.620237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:41.939854 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:41.939827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:50.854705 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:50.854667    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:51.374595 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:51.374561    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:51.628446 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:51.628375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:54.327420 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.327377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:54.403323 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.403288    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"]
Jul 16 22:31:54.403692 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.403678    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:31:54.403763 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.403693    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:31:54.403763 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.403705    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:31:54.403763 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.403711    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:31:54.403940 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.403784    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:31:54.403940 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.403793    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:31:54.406950 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.406904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.409890 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.409870    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f5e7ed86-serving-cert\""
Jul 16 22:31:54.419831 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.419808    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"]
Jul 16 22:31:54.452309 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.452274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:54.484579 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.484544    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-maas-api-tls\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.484725 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.484590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-openshift-service-ca\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.484725 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.484666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k54w5\" (UniqueName: \"kubernetes.io/projected/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-kube-api-access-k54w5\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.585648 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.585570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-maas-api-tls\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.585648 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.585621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-openshift-service-ca\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.585835 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.585658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k54w5\" (UniqueName: \"kubernetes.io/projected/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-kube-api-access-k54w5\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.586391 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.586362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-openshift-service-ca\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.588191 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.588171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-maas-api-tls\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.598148 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.598126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:54.603593 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.603576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k54w5\" (UniqueName: \"kubernetes.io/projected/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-kube-api-access-k54w5\") pod \"maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.718242 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.718208    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:54.729347 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.729310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:54.875176 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:54.875152    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"]
Jul 16 22:31:54.877133 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:31:54.877096    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e43deb_33f2_4744_aaa2_d5cf8d5bfa94.slice/crio-037d082a9602a68713d51f72be5f32449bd25bd5d8baa9dbc2232fb4117a9256 WatchSource:0}: Error finding container 037d082a9602a68713d51f72be5f32449bd25bd5d8baa9dbc2232fb4117a9256: Status 404 returned error can't find the container with id 037d082a9602a68713d51f72be5f32449bd25bd5d8baa9dbc2232fb4117a9256
Jul 16 22:31:55.180981 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:55.180875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:55.312777 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:55.312749    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7" event={"ID":"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94","Type":"ContainerStarted","Data":"037d082a9602a68713d51f72be5f32449bd25bd5d8baa9dbc2232fb4117a9256"}
Jul 16 22:31:55.414531 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:55.414496    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:56.319298 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:56.319267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7" event={"ID":"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94","Type":"ContainerStarted","Data":"76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9"}
Jul 16 22:31:56.319463 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:56.319392    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:31:56.375998 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:56.375948    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7" podStartSLOduration=2.009449838 podStartE2EDuration="2.375932777s" totalImagesPullingTime="366.482939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:31:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:31:56.37393183 +0000 UTC m=+2582.763205629" watchObservedRunningTime="2026-07-16 22:31:56.375932777 +0000 UTC m=+2582.765206554"
Jul 16 22:31:58.910975 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:31:58.910935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:02.331322 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:02.331292    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:32:06.596586 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.596545    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"]
Jul 16 22:32:06.597013 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.596987    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:32:06.597013 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.597004    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:32:06.597147 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.597133    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ffac2e-6759-40d8-9122-fe214d723f1c" containerName="revoke-keys"
Jul 16 22:32:06.600704 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.600681    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.609896 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.609875    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f5e7ed86\"/\"default-dockercfg-tmwlz\""
Jul 16 22:32:06.610038 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.609966    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f5e7ed86\"/\"kube-root-ca.crt\""
Jul 16 22:32:06.610900 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.610884    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f5e7ed86\"/\"openshift-service-ca.crt\""
Jul 16 22:32:06.611539 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.611522    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f5e7ed86\"/\"e2e-lifecycle-model-f5e7ed86-kserve-self-signed-certs\""
Jul 16 22:32:06.626432 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.626407    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"]
Jul 16 22:32:06.633303 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.633279    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:06.688746 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.688712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-tmp-dir\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.688746 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.688743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b9685e8-de77-4139-b1b3-8954e39e825f-tls-certs\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.688973 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.688835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-model-cache\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.688973 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.688873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-home\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.688973 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.688888    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-dshm\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.688973 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.688929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv249\" (UniqueName: \"kubernetes.io/projected/9b9685e8-de77-4139-b1b3-8954e39e825f-kube-api-access-jv249\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790018 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.789978    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-tmp-dir\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790018 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b9685e8-de77-4139-b1b3-8954e39e825f-tls-certs\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790227 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-model-cache\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790227 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-home\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790227 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-dshm\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790382 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv249\" (UniqueName: \"kubernetes.io/projected/9b9685e8-de77-4139-b1b3-8954e39e825f-kube-api-access-jv249\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790498 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-model-cache\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790498    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-tmp-dir\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.790568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.790537    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-home\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.792453 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.792429    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-dshm\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.792678 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.792648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b9685e8-de77-4139-b1b3-8954e39e825f-tls-certs\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.807317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.807289    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv249\" (UniqueName: \"kubernetes.io/projected/9b9685e8-de77-4139-b1b3-8954e39e825f-kube-api-access-jv249\") pod \"e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") " pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:06.910375 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:06.910296    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:07.068696 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:07.068655    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"]
Jul 16 22:32:07.071119 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:32:07.071091    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b9685e8_de77_4139_b1b3_8954e39e825f.slice/crio-3151ba99454fc0488239507348d48f2da3268d14c2e9d6e775ef22158ca489ee WatchSource:0}: Error finding container 3151ba99454fc0488239507348d48f2da3268d14c2e9d6e775ef22158ca489ee: Status 404 returned error can't find the container with id 3151ba99454fc0488239507348d48f2da3268d14c2e9d6e775ef22158ca489ee
Jul 16 22:32:07.379802 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:07.379762    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc" event={"ID":"9b9685e8-de77-4139-b1b3-8954e39e825f","Type":"ContainerStarted","Data":"865ff1769a2929c511f904a76b7be335b83738cda60f2f48444b72ac7469ae48"}
Jul 16 22:32:07.380017 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:07.379816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc" event={"ID":"9b9685e8-de77-4139-b1b3-8954e39e825f","Type":"ContainerStarted","Data":"3151ba99454fc0488239507348d48f2da3268d14c2e9d6e775ef22158ca489ee"}
Jul 16 22:32:07.399549 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:07.399496    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc" podStartSLOduration=1.399481384 podStartE2EDuration="1.399481384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:32:07.39778806 +0000 UTC m=+2593.787061871" watchObservedRunningTime="2026-07-16 22:32:07.399481384 +0000 UTC m=+2593.788755169"
Jul 16 22:32:16.911052 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:16.910967    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:16.911052 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:16.911021    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:16.924019 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:16.923992    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:17.440632 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:17.440602    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:32:37.951146 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:37.951114    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"]
Jul 16 22:32:37.951657 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:37.951336    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7" podUID="93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" containerName="maas-api" containerID="cri-o://76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9" gracePeriod=30
Jul 16 22:32:38.011109 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.011075    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:38.209840 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.209785    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:32:38.270385 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.270354    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-maas-api-tls\") pod \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") "
Jul 16 22:32:38.270550 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.270406    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-openshift-service-ca\") pod \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") "
Jul 16 22:32:38.270550 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.270531    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k54w5\" (UniqueName: \"kubernetes.io/projected/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-kube-api-access-k54w5\") pod \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\" (UID: \"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94\") "
Jul 16 22:32:38.270730 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.270712    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" (UID: "93e43deb-33f2-4744-aaa2-d5cf8d5bfa94"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:32:38.270860 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.270845    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:32:38.272513 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.272492    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" (UID: "93e43deb-33f2-4744-aaa2-d5cf8d5bfa94"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:32:38.272677 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.272663    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-kube-api-access-k54w5" (OuterVolumeSpecName: "kube-api-access-k54w5") pod "93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" (UID: "93e43deb-33f2-4744-aaa2-d5cf8d5bfa94"). InnerVolumeSpecName "kube-api-access-k54w5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:32:38.371403 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.371371    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:32:38.371403 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.371405    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k54w5\" (UniqueName: \"kubernetes.io/projected/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94-kube-api-access-k54w5\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:32:38.535379 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.535294    2564 generic.go:356] "Generic (PLEG): container finished" podID="93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" containerID="76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9" exitCode=0
Jul 16 22:32:38.535379 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.535342    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"
Jul 16 22:32:38.535563 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.535374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7" event={"ID":"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94","Type":"ContainerDied","Data":"76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9"}
Jul 16 22:32:38.535563 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.535425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7" event={"ID":"93e43deb-33f2-4744-aaa2-d5cf8d5bfa94","Type":"ContainerDied","Data":"037d082a9602a68713d51f72be5f32449bd25bd5d8baa9dbc2232fb4117a9256"}
Jul 16 22:32:38.535563 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.535447    2564 scope.go:117] "RemoveContainer" containerID="76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9"
Jul 16 22:32:38.545111 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.545092    2564 scope.go:117] "RemoveContainer" containerID="76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9"
Jul 16 22:32:38.545376 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:32:38.545357    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9\": container with ID starting with 76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9 not found: ID does not exist" containerID="76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9"
Jul 16 22:32:38.545441 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.545385    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9"} err="failed to get container status \"76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9\": rpc error: code = NotFound desc = could not find container \"76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9\": container with ID starting with 76b10054c81829de42a5a43589ad5ca3bc31b34c4fc860d4ffc2094389ec8da9 not found: ID does not exist"
Jul 16 22:32:38.559039 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.559016    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"]
Jul 16 22:32:38.564348 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:38.564329    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f5e7ed86-78ddc485bd-xgmr7"]
Jul 16 22:32:40.125412 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:40.125379    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" path="/var/lib/kubelet/pods/93e43deb-33f2-4744-aaa2-d5cf8d5bfa94/volumes"
Jul 16 22:32:42.455958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:42.455900    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:43.149651 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:43.149606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:43.252391 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:43.252359    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:47.434934 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:47.434882    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"]
Jul 16 22:32:47.529563 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:47.529533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:50.927431 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.927391    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"]
Jul 16 22:32:50.927814 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.927788    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" containerName="maas-api"
Jul 16 22:32:50.927814 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.927799    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" containerName="maas-api"
Jul 16 22:32:50.927888 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.927867    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="93e43deb-33f2-4744-aaa2-d5cf8d5bfa94" containerName="maas-api"
Jul 16 22:32:50.932114 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.932096    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"
Jul 16 22:32:50.934612 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.934589    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:32:50.938122 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.938099    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"]
Jul 16 22:32:50.993016 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:50.992978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b3d5a0-008a-4084-a712-f5a19b4d3067-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-bslg6\" (UID: \"a1b3d5a0-008a-4084-a712-f5a19b4d3067\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"
Jul 16 22:32:51.093886 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:51.093847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b3d5a0-008a-4084-a712-f5a19b4d3067-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-bslg6\" (UID: \"a1b3d5a0-008a-4084-a712-f5a19b4d3067\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"
Jul 16 22:32:51.094455 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:51.094436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b3d5a0-008a-4084-a712-f5a19b4d3067-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-bslg6\" (UID: \"a1b3d5a0-008a-4084-a712-f5a19b4d3067\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"
Jul 16 22:32:51.244234 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:51.244203    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"
Jul 16 22:32:51.377938 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:51.377889    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"]
Jul 16 22:32:51.380688 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:32:51.380663    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b3d5a0_008a_4084_a712_f5a19b4d3067.slice/crio-db6aea81384086a14f7bc82dc333bab3bc02e146617aa19118996c4af484eca1 WatchSource:0}: Error finding container db6aea81384086a14f7bc82dc333bab3bc02e146617aa19118996c4af484eca1: Status 404 returned error can't find the container with id db6aea81384086a14f7bc82dc333bab3bc02e146617aa19118996c4af484eca1
Jul 16 22:32:51.609652 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:51.609619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" event={"ID":"a1b3d5a0-008a-4084-a712-f5a19b4d3067","Type":"ContainerStarted","Data":"db6aea81384086a14f7bc82dc333bab3bc02e146617aa19118996c4af484eca1"}
Jul 16 22:32:52.616536 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:52.616493    2564 generic.go:356] "Generic (PLEG): container finished" podID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerID="dbbdf500347485dad56660f792e1a0f6bb844b9914309756d892d4e80b5e2c70" exitCode=6
Jul 16 22:32:52.616990 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:52.616582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" event={"ID":"a1b3d5a0-008a-4084-a712-f5a19b4d3067","Type":"ContainerDied","Data":"dbbdf500347485dad56660f792e1a0f6bb844b9914309756d892d4e80b5e2c70"}
Jul 16 22:32:52.616990 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:52.616747    2564 scope.go:117] "RemoveContainer" containerID="dbbdf500347485dad56660f792e1a0f6bb844b9914309756d892d4e80b5e2c70"
Jul 16 22:32:53.626736 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:53.626643    2564 generic.go:356] "Generic (PLEG): container finished" podID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerID="4302896de9cfa238f38e2568782afc8f529d293a80dd25d4370c830a99f75fd1" exitCode=6
Jul 16 22:32:53.627167 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:53.626737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" event={"ID":"a1b3d5a0-008a-4084-a712-f5a19b4d3067","Type":"ContainerDied","Data":"4302896de9cfa238f38e2568782afc8f529d293a80dd25d4370c830a99f75fd1"}
Jul 16 22:32:53.627167 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:53.626800    2564 scope.go:117] "RemoveContainer" containerID="dbbdf500347485dad56660f792e1a0f6bb844b9914309756d892d4e80b5e2c70"
Jul 16 22:32:53.627167 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:53.627076    2564 scope.go:117] "RemoveContainer" containerID="4302896de9cfa238f38e2568782afc8f529d293a80dd25d4370c830a99f75fd1"
Jul 16 22:32:53.627329 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:32:53.627310    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-bslg6_odh-ai-gateway-infra(a1b3d5a0-008a-4084-a712-f5a19b4d3067)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067"
Jul 16 22:32:54.633649 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:32:54.633623    2564 scope.go:117] "RemoveContainer" containerID="4302896de9cfa238f38e2568782afc8f529d293a80dd25d4370c830a99f75fd1"
Jul 16 22:32:54.634062 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:32:54.633796    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-bslg6_odh-ai-gateway-infra(a1b3d5a0-008a-4084-a712-f5a19b4d3067)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067"
Jul 16 22:33:02.455990 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.455944    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc" podUID="9b9685e8-de77-4139-b1b3-8954e39e825f" containerName="main" containerID="cri-o://865ff1769a2929c511f904a76b7be335b83738cda60f2f48444b72ac7469ae48" gracePeriod=45
Jul 16 22:33:02.674826 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.674770    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b9685e8-de77-4139-b1b3-8954e39e825f" containerID="865ff1769a2929c511f904a76b7be335b83738cda60f2f48444b72ac7469ae48" exitCode=0
Jul 16 22:33:02.674826 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.674820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc" event={"ID":"9b9685e8-de77-4139-b1b3-8954e39e825f","Type":"ContainerDied","Data":"865ff1769a2929c511f904a76b7be335b83738cda60f2f48444b72ac7469ae48"}
Jul 16 22:33:02.708464 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.708405    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:33:02.796950 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.796920    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-dshm\") pod \"9b9685e8-de77-4139-b1b3-8954e39e825f\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") "
Jul 16 22:33:02.797124 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797005    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-model-cache\") pod \"9b9685e8-de77-4139-b1b3-8954e39e825f\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") "
Jul 16 22:33:02.797124 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797035    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv249\" (UniqueName: \"kubernetes.io/projected/9b9685e8-de77-4139-b1b3-8954e39e825f-kube-api-access-jv249\") pod \"9b9685e8-de77-4139-b1b3-8954e39e825f\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") "
Jul 16 22:33:02.797124 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797074    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-home\") pod \"9b9685e8-de77-4139-b1b3-8954e39e825f\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") "
Jul 16 22:33:02.797124 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797114    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b9685e8-de77-4139-b1b3-8954e39e825f-tls-certs\") pod \"9b9685e8-de77-4139-b1b3-8954e39e825f\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") "
Jul 16 22:33:02.797364 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797144    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-tmp-dir\") pod \"9b9685e8-de77-4139-b1b3-8954e39e825f\" (UID: \"9b9685e8-de77-4139-b1b3-8954e39e825f\") "
Jul 16 22:33:02.797419 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797380    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-home" (OuterVolumeSpecName: "home") pod "9b9685e8-de77-4139-b1b3-8954e39e825f" (UID: "9b9685e8-de77-4139-b1b3-8954e39e825f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:33:02.797564 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797537    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b9685e8-de77-4139-b1b3-8954e39e825f" (UID: "9b9685e8-de77-4139-b1b3-8954e39e825f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:33:02.797712 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.797653    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b9685e8-de77-4139-b1b3-8954e39e825f" (UID: "9b9685e8-de77-4139-b1b3-8954e39e825f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:33:02.799516 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.799492    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b9685e8-de77-4139-b1b3-8954e39e825f-kube-api-access-jv249" (OuterVolumeSpecName: "kube-api-access-jv249") pod "9b9685e8-de77-4139-b1b3-8954e39e825f" (UID: "9b9685e8-de77-4139-b1b3-8954e39e825f"). InnerVolumeSpecName "kube-api-access-jv249". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:33:02.799759 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.799740    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-dshm" (OuterVolumeSpecName: "dshm") pod "9b9685e8-de77-4139-b1b3-8954e39e825f" (UID: "9b9685e8-de77-4139-b1b3-8954e39e825f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:33:02.799834 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.799757    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b9685e8-de77-4139-b1b3-8954e39e825f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b9685e8-de77-4139-b1b3-8954e39e825f" (UID: "9b9685e8-de77-4139-b1b3-8954e39e825f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:33:02.898888 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.898849    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:33:02.898888 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.898881    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv249\" (UniqueName: \"kubernetes.io/projected/9b9685e8-de77-4139-b1b3-8954e39e825f-kube-api-access-jv249\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:33:02.898888 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.898892    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:33:02.899137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.898901    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b9685e8-de77-4139-b1b3-8954e39e825f-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:33:02.899137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.898937    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:33:02.899137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:02.898944    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b9685e8-de77-4139-b1b3-8954e39e825f-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:33:03.681703 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:03.681668    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"
Jul 16 22:33:03.682153 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:03.681715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc" event={"ID":"9b9685e8-de77-4139-b1b3-8954e39e825f","Type":"ContainerDied","Data":"3151ba99454fc0488239507348d48f2da3268d14c2e9d6e775ef22158ca489ee"}
Jul 16 22:33:03.682153 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:03.681764    2564 scope.go:117] "RemoveContainer" containerID="865ff1769a2929c511f904a76b7be335b83738cda60f2f48444b72ac7469ae48"
Jul 16 22:33:03.708147 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:03.708111    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"]
Jul 16 22:33:03.711690 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:03.711666    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f5e7ed86/e2e-lifecycle-model-f5e7ed86-kserve-85bbd79c9f-mjgmc"]
Jul 16 22:33:04.125903 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:04.125867    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b9685e8-de77-4139-b1b3-8954e39e825f" path="/var/lib/kubelet/pods/9b9685e8-de77-4139-b1b3-8954e39e825f/volumes"
Jul 16 22:33:09.120144 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:09.120120    2564 scope.go:117] "RemoveContainer" containerID="4302896de9cfa238f38e2568782afc8f529d293a80dd25d4370c830a99f75fd1"
Jul 16 22:33:09.715084 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:09.715048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" event={"ID":"a1b3d5a0-008a-4084-a712-f5a19b4d3067","Type":"ContainerStarted","Data":"6e3cfee2992f4ee9705cf08db8490875f3e651c9bb9d19b60994b436f6fe895f"}
Jul 16 22:33:09.733944 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:09.733875    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" podStartSLOduration=19.733858288 podStartE2EDuration="19.733858288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:33:09.732092982 +0000 UTC m=+2656.121366767" watchObservedRunningTime="2026-07-16 22:33:09.733858288 +0000 UTC m=+2656.123132074"
Jul 16 22:33:10.722307 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:10.722272    2564 generic.go:356] "Generic (PLEG): container finished" podID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerID="6e3cfee2992f4ee9705cf08db8490875f3e651c9bb9d19b60994b436f6fe895f" exitCode=6
Jul 16 22:33:10.722740 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:10.722345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" event={"ID":"a1b3d5a0-008a-4084-a712-f5a19b4d3067","Type":"ContainerDied","Data":"6e3cfee2992f4ee9705cf08db8490875f3e651c9bb9d19b60994b436f6fe895f"}
Jul 16 22:33:10.722740 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:10.722406    2564 scope.go:117] "RemoveContainer" containerID="4302896de9cfa238f38e2568782afc8f529d293a80dd25d4370c830a99f75fd1"
Jul 16 22:33:10.722822 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:10.722758    2564 scope.go:117] "RemoveContainer" containerID="6e3cfee2992f4ee9705cf08db8490875f3e651c9bb9d19b60994b436f6fe895f"
Jul 16 22:33:10.723026 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:33:10.723007    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-bslg6_odh-ai-gateway-infra(a1b3d5a0-008a-4084-a712-f5a19b4d3067)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067"
Jul 16 22:33:10.749477 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:10.749444    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"]
Jul 16 22:33:11.871672 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:11.871644    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"
Jul 16 22:33:11.985098 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:11.985001    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b3d5a0-008a-4084-a712-f5a19b4d3067-maas-api-service-ca\") pod \"a1b3d5a0-008a-4084-a712-f5a19b4d3067\" (UID: \"a1b3d5a0-008a-4084-a712-f5a19b4d3067\") "
Jul 16 22:33:11.985372 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:11.985347    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b3d5a0-008a-4084-a712-f5a19b4d3067-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a1b3d5a0-008a-4084-a712-f5a19b4d3067" (UID: "a1b3d5a0-008a-4084-a712-f5a19b4d3067"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:33:12.086029 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:12.085992    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1b3d5a0-008a-4084-a712-f5a19b4d3067-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:33:12.736731 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:12.736693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6" event={"ID":"a1b3d5a0-008a-4084-a712-f5a19b4d3067","Type":"ContainerDied","Data":"db6aea81384086a14f7bc82dc333bab3bc02e146617aa19118996c4af484eca1"}
Jul 16 22:33:12.736731 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:12.736731    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"
Jul 16 22:33:12.736979 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:12.736770    2564 scope.go:117] "RemoveContainer" containerID="6e3cfee2992f4ee9705cf08db8490875f3e651c9bb9d19b60994b436f6fe895f"
Jul 16 22:33:12.757361 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:12.757333    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"]
Jul 16 22:33:12.767333 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:12.767302    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-bslg6"]
Jul 16 22:33:14.130529 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:14.130490    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" path="/var/lib/kubelet/pods/a1b3d5a0-008a-4084-a712-f5a19b4d3067/volumes"
Jul 16 22:33:23.673688 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:23.673650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:24.932170 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:24.932133    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:30.855805 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:30.855771    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:46.292248 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:46.292168    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:46.418471 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:46.418434    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:50.989544 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.989512    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"]
Jul 16 22:33:50.989958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.989886    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9685e8-de77-4139-b1b3-8954e39e825f" containerName="main"
Jul 16 22:33:50.989958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.989897    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9685e8-de77-4139-b1b3-8954e39e825f" containerName="main"
Jul 16 22:33:50.989958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.989922    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:33:50.989958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.989927    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:33:50.989958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.989939    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:33:50.989958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.989945    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:33:50.990158 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.990004    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:33:50.990158 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.990013    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:33:50.990158 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.990020    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:33:50.990158 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.990028    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b9685e8-de77-4139-b1b3-8954e39e825f" containerName="main"
Jul 16 22:33:50.993100 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.993084    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"
Jul 16 22:33:50.995474 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:50.995453    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:33:51.000144 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.000123    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"]
Jul 16 22:33:51.117343 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.117312    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea64a026-02f2-4372-b457-c395beee8400-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-2wpxb\" (UID: \"ea64a026-02f2-4372-b457-c395beee8400\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"
Jul 16 22:33:51.218809 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.218760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea64a026-02f2-4372-b457-c395beee8400-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-2wpxb\" (UID: \"ea64a026-02f2-4372-b457-c395beee8400\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"
Jul 16 22:33:51.219419 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.219400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea64a026-02f2-4372-b457-c395beee8400-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23b64857-2wpxb\" (UID: \"ea64a026-02f2-4372-b457-c395beee8400\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"
Jul 16 22:33:51.305554 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.305472    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"
Jul 16 22:33:51.441292 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.441261    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"]
Jul 16 22:33:51.444628 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:33:51.444594    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea64a026_02f2_4372_b457_c395beee8400.slice/crio-34a45121112d48d4e42e60c282b93cd911a12939a5e9210b19b03b3436002876 WatchSource:0}: Error finding container 34a45121112d48d4e42e60c282b93cd911a12939a5e9210b19b03b3436002876: Status 404 returned error can't find the container with id 34a45121112d48d4e42e60c282b93cd911a12939a5e9210b19b03b3436002876
Jul 16 22:33:51.935040 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.934956    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" event={"ID":"ea64a026-02f2-4372-b457-c395beee8400","Type":"ContainerStarted","Data":"86e4cde352fd51bdacd171a440b2bb61081cc51ab614edcb73bd743824736a07"}
Jul 16 22:33:51.935040 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.935013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" event={"ID":"ea64a026-02f2-4372-b457-c395beee8400","Type":"ContainerStarted","Data":"34a45121112d48d4e42e60c282b93cd911a12939a5e9210b19b03b3436002876"}
Jul 16 22:33:51.951303 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:51.951258    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" podStartSLOduration=1.95124325 podStartE2EDuration="1.95124325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:33:51.950857195 +0000 UTC m=+2698.340130982" watchObservedRunningTime="2026-07-16 22:33:51.95124325 +0000 UTC m=+2698.340517035"
Jul 16 22:33:52.942369 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:52.942329    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea64a026-02f2-4372-b457-c395beee8400" containerID="86e4cde352fd51bdacd171a440b2bb61081cc51ab614edcb73bd743824736a07" exitCode=6
Jul 16 22:33:52.942852 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:52.942412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" event={"ID":"ea64a026-02f2-4372-b457-c395beee8400","Type":"ContainerDied","Data":"86e4cde352fd51bdacd171a440b2bb61081cc51ab614edcb73bd743824736a07"}
Jul 16 22:33:52.942852 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:52.942636    2564 scope.go:117] "RemoveContainer" containerID="86e4cde352fd51bdacd171a440b2bb61081cc51ab614edcb73bd743824736a07"
Jul 16 22:33:53.949843 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:53.949806    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea64a026-02f2-4372-b457-c395beee8400" containerID="6ea1378e0b9c67074846e8a94d1e65987ee19ca89f7b05877f23a7dc9c9f820e" exitCode=6
Jul 16 22:33:53.950251 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:53.949894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" event={"ID":"ea64a026-02f2-4372-b457-c395beee8400","Type":"ContainerDied","Data":"6ea1378e0b9c67074846e8a94d1e65987ee19ca89f7b05877f23a7dc9c9f820e"}
Jul 16 22:33:53.950251 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:53.949962    2564 scope.go:117] "RemoveContainer" containerID="86e4cde352fd51bdacd171a440b2bb61081cc51ab614edcb73bd743824736a07"
Jul 16 22:33:53.950251 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:53.950175    2564 scope.go:117] "RemoveContainer" containerID="6ea1378e0b9c67074846e8a94d1e65987ee19ca89f7b05877f23a7dc9c9f820e"
Jul 16 22:33:53.950428 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:33:53.950408    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-2wpxb_odh-ai-gateway-infra(ea64a026-02f2-4372-b457-c395beee8400)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" podUID="ea64a026-02f2-4372-b457-c395beee8400"
Jul 16 22:33:54.265630 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:54.265540    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:33:54.271707 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:54.271675    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:33:54.274539 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:54.274521    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:33:54.279673 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:54.279657    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:33:54.951411 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:54.951385    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:54.957166 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:54.957145    2564 scope.go:117] "RemoveContainer" containerID="6ea1378e0b9c67074846e8a94d1e65987ee19ca89f7b05877f23a7dc9c9f820e"
Jul 16 22:33:54.957355 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:33:54.957337    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-2wpxb_odh-ai-gateway-infra(ea64a026-02f2-4372-b457-c395beee8400)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" podUID="ea64a026-02f2-4372-b457-c395beee8400"
Jul 16 22:33:55.194409 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:55.194295    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:55.395493 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:55.395459    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:56.323507 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:56.323466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:56.425058 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:56.425020    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:57.330412 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:57.330379    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:57.430982 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:57.430949    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:57.528171 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:57.528132    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:33:59.025189 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:33:59.025154    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:05.481331 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:05.481301    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:05.566525 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:05.566489    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:06.189275 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:06.189245    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:08.119724 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:08.119692    2564 scope.go:117] "RemoveContainer" containerID="6ea1378e0b9c67074846e8a94d1e65987ee19ca89f7b05877f23a7dc9c9f820e"
Jul 16 22:34:09.035267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:09.035182    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea64a026-02f2-4372-b457-c395beee8400" containerID="b3893fc7a971728c9cc62e2591139eec20abca089a8480e2a7e60176e387ba70" exitCode=6
Jul 16 22:34:09.035410 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:09.035263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" event={"ID":"ea64a026-02f2-4372-b457-c395beee8400","Type":"ContainerDied","Data":"b3893fc7a971728c9cc62e2591139eec20abca089a8480e2a7e60176e387ba70"}
Jul 16 22:34:09.035410 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:09.035330    2564 scope.go:117] "RemoveContainer" containerID="6ea1378e0b9c67074846e8a94d1e65987ee19ca89f7b05877f23a7dc9c9f820e"
Jul 16 22:34:09.035669 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:09.035652    2564 scope.go:117] "RemoveContainer" containerID="b3893fc7a971728c9cc62e2591139eec20abca089a8480e2a7e60176e387ba70"
Jul 16 22:34:09.036023 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:34:09.035980    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23b64857-2wpxb_odh-ai-gateway-infra(ea64a026-02f2-4372-b457-c395beee8400)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" podUID="ea64a026-02f2-4372-b457-c395beee8400"
Jul 16 22:34:10.061031 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:10.060997    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"]
Jul 16 22:34:10.199342 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:10.199316    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"
Jul 16 22:34:10.280608 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:10.280573    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea64a026-02f2-4372-b457-c395beee8400-maas-api-service-ca\") pod \"ea64a026-02f2-4372-b457-c395beee8400\" (UID: \"ea64a026-02f2-4372-b457-c395beee8400\") "
Jul 16 22:34:10.281010 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:10.280983    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea64a026-02f2-4372-b457-c395beee8400-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea64a026-02f2-4372-b457-c395beee8400" (UID: "ea64a026-02f2-4372-b457-c395beee8400"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:34:10.381674 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:10.381590    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea64a026-02f2-4372-b457-c395beee8400-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:34:11.048382 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:11.048347    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb" event={"ID":"ea64a026-02f2-4372-b457-c395beee8400","Type":"ContainerDied","Data":"34a45121112d48d4e42e60c282b93cd911a12939a5e9210b19b03b3436002876"}
Jul 16 22:34:11.048580 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:11.048429    2564 scope.go:117] "RemoveContainer" containerID="b3893fc7a971728c9cc62e2591139eec20abca089a8480e2a7e60176e387ba70"
Jul 16 22:34:11.048580 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:11.048356    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"
Jul 16 22:34:11.072996 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:11.072963    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"]
Jul 16 22:34:11.077384 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:11.077360    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23b64857-2wpxb"]
Jul 16 22:34:11.983149 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:11.983119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:12.091152 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:12.091114    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:12.126348 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:12.126318    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea64a026-02f2-4372-b457-c395beee8400" path="/var/lib/kubelet/pods/ea64a026-02f2-4372-b457-c395beee8400/volumes"
Jul 16 22:34:12.191408 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:12.191362    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:22.792564 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:22.792515    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:23.114693 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:23.114613    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:44.118417 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:44.118374    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:50.798584 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:50.798552    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:51.083604 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:51.083521    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:51.201610 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:51.201557    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:51.570832 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:51.570796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:51.802111 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:51.802067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:55.126156 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.126112    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:55.226225 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.226110    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:55.401365 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401331    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"]
Jul 16 22:34:55.401787 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401768    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.401876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401789    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.401876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401806    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.401876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401813    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.401876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401840    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.401876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401848    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.401876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401869    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:34:55.401876 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401876    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b3d5a0-008a-4084-a712-f5a19b4d3067" containerName="revoke-keys"
Jul 16 22:34:55.402244 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.401988    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.402244 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.402002    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:34:55.406433 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.406411    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.409807 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.409783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fb6c08-serving-cert\""
Jul 16 22:34:55.418503 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.418481    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"]
Jul 16 22:34:55.453569 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.453537    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:55.551658 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.551580    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mfq7\" (UniqueName: \"kubernetes.io/projected/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-kube-api-access-2mfq7\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.551658 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.551624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.551841 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.551707    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.653103 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.653064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.653284 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.653128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mfq7\" (UniqueName: \"kubernetes.io/projected/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-kube-api-access-2mfq7\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.653284 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.653154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.653284 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:34:55.653216    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-serving-cert: secret "maas-api-e2e-inf-a-fb6c08-serving-cert" not found
Jul 16 22:34:55.653408 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:34:55.653298    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls podName:2ce925cf-a3ef-4d24-8cbb-59177bd81e7a nodeName:}" failed. No retries permitted until 2026-07-16 22:34:56.153279804 +0000 UTC m=+2762.542553569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls") pod "maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2" (UID: "2ce925cf-a3ef-4d24-8cbb-59177bd81e7a") : secret "maas-api-e2e-inf-a-fb6c08-serving-cert" not found
Jul 16 22:34:55.653733 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.653716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.674736 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.674708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mfq7\" (UniqueName: \"kubernetes.io/projected/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-kube-api-access-2mfq7\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:55.910955 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:55.910849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:56.033232 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:56.033197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:56.134585 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:56.134549    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:56.156642 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:56.156610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:56.159125 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:56.159098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:56.317822 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:56.317793    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:56.467393 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:56.467362    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"]
Jul 16 22:34:56.469335 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:34:56.469307    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce925cf_a3ef_4d24_8cbb_59177bd81e7a.slice/crio-8926cf42aa95751036974c85267e8e9b50b28acd3071aaa050943807ab47c53a WatchSource:0}: Error finding container 8926cf42aa95751036974c85267e8e9b50b28acd3071aaa050943807ab47c53a: Status 404 returned error can't find the container with id 8926cf42aa95751036974c85267e8e9b50b28acd3071aaa050943807ab47c53a
Jul 16 22:34:57.284328 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:57.284289    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2" event={"ID":"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a","Type":"ContainerStarted","Data":"6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c"}
Jul 16 22:34:57.284792 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:57.284353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2" event={"ID":"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a","Type":"ContainerStarted","Data":"8926cf42aa95751036974c85267e8e9b50b28acd3071aaa050943807ab47c53a"}
Jul 16 22:34:57.284792 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:57.284383    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:34:57.307365 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:57.307310    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2" podStartSLOduration=1.689739908 podStartE2EDuration="2.307296266s" totalImagesPullingTime="617.556358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:34:57.305476575 +0000 UTC m=+2763.694750361" watchObservedRunningTime="2026-07-16 22:34:57.307296266 +0000 UTC m=+2763.696570051"
Jul 16 22:34:59.679977 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:59.679897    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:34:59.998785 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:34:59.998754    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:00.354770 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:00.354640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:00.631517 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:00.631417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:03.296166 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.296127    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:35:03.301201 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.301175    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"]
Jul 16 22:35:03.301614 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.301601    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea64a026-02f2-4372-b457-c395beee8400" containerName="revoke-keys"
Jul 16 22:35:03.306350 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.306319    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.312670 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.312630    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-2dbd48-serving-cert\""
Jul 16 22:35:03.371254 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.371222    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"]
Jul 16 22:35:03.410890 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.410852    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:03.417067 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.417036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/765a4764-08ea-405d-9c5b-d6190ad63949-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.417215 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.417099    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.417277 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.417216    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx4fw\" (UniqueName: \"kubernetes.io/projected/765a4764-08ea-405d-9c5b-d6190ad63949-kube-api-access-rx4fw\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.518014 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.517975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.518207 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.518058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx4fw\" (UniqueName: \"kubernetes.io/projected/765a4764-08ea-405d-9c5b-d6190ad63949-kube-api-access-rx4fw\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.518207 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.518111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/765a4764-08ea-405d-9c5b-d6190ad63949-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.518207 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:35:03.518143    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-serving-cert: secret "maas-api-e2e-inf-b-2dbd48-serving-cert" not found
Jul 16 22:35:03.518377 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:35:03.518218    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls podName:765a4764-08ea-405d-9c5b-d6190ad63949 nodeName:}" failed. No retries permitted until 2026-07-16 22:35:04.018195869 +0000 UTC m=+2770.407469650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls") pod "maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn" (UID: "765a4764-08ea-405d-9c5b-d6190ad63949") : secret "maas-api-e2e-inf-b-2dbd48-serving-cert" not found
Jul 16 22:35:03.518693 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.518676    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/765a4764-08ea-405d-9c5b-d6190ad63949-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.531121 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.531089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx4fw\" (UniqueName: \"kubernetes.io/projected/765a4764-08ea-405d-9c5b-d6190ad63949-kube-api-access-rx4fw\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:03.547031 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:03.546929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:04.021647 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:04.021613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:04.024547 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:04.024507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls\") pod \"maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:04.150035 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:04.150002    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:04.221482 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:04.221453    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:04.259594 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:04.259564    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:04.366449 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:04.366418    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:04.378818 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:04.378792    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"]
Jul 16 22:35:04.380818 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:35:04.380793    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod765a4764_08ea_405d_9c5b_d6190ad63949.slice/crio-edb98dbebd12317bf780680cc4e2dbb5d1a5afe8bb99def13cb0624cc82d6aa9 WatchSource:0}: Error finding container edb98dbebd12317bf780680cc4e2dbb5d1a5afe8bb99def13cb0624cc82d6aa9: Status 404 returned error can't find the container with id edb98dbebd12317bf780680cc4e2dbb5d1a5afe8bb99def13cb0624cc82d6aa9
Jul 16 22:35:05.334335 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:05.334298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn" event={"ID":"765a4764-08ea-405d-9c5b-d6190ad63949","Type":"ContainerStarted","Data":"edb98dbebd12317bf780680cc4e2dbb5d1a5afe8bb99def13cb0624cc82d6aa9"}
Jul 16 22:35:05.770118 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:05.770074    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:06.341226 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:06.341190    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn" event={"ID":"765a4764-08ea-405d-9c5b-d6190ad63949","Type":"ContainerStarted","Data":"9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb"}
Jul 16 22:35:06.341429 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:06.341388    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:06.363017 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:06.362961    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn" podStartSLOduration=2.437056121 podStartE2EDuration="3.362946063s" totalImagesPullingTime="925.889942ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:35:06.361729542 +0000 UTC m=+2772.751003346" watchObservedRunningTime="2026-07-16 22:35:06.362946063 +0000 UTC m=+2772.752219851"
Jul 16 22:35:07.618622 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:07.618591    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:12.352341 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:12.352312    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:35:16.876491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:16.876393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:16.969902 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:16.969864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:25.209317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:25.209282    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:25.310811 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:25.310776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:26.714127 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:26.714068    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:35:51.160212 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:35:51.160180    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:08.501950 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:08.501893    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:08.605427 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:08.605397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:16.811535 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:16.811485    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:17.795861 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:17.795834    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:18.520097 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:18.520067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:18.622821 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:18.622755    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:28.488795 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.488754    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"]
Jul 16 22:36:28.489454 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.489048    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2" podUID="2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" containerName="maas-api" containerID="cri-o://6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c" gracePeriod=30
Jul 16 22:36:28.519826 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.519795    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:28.950338 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.950312    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:36:28.954979 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.954958    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mfq7\" (UniqueName: \"kubernetes.io/projected/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-kube-api-access-2mfq7\") pod \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") "
Jul 16 22:36:28.955097 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.955045    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-openshift-service-ca\") pod \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") "
Jul 16 22:36:28.955097 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.955062    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls\") pod \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\" (UID: \"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a\") "
Jul 16 22:36:28.955365 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.955345    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" (UID: "2ce925cf-a3ef-4d24-8cbb-59177bd81e7a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:36:28.957167 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.957146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" (UID: "2ce925cf-a3ef-4d24-8cbb-59177bd81e7a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:36:28.957236 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:28.957192    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-kube-api-access-2mfq7" (OuterVolumeSpecName: "kube-api-access-2mfq7") pod "2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" (UID: "2ce925cf-a3ef-4d24-8cbb-59177bd81e7a"). InnerVolumeSpecName "kube-api-access-2mfq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:36:29.055687 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.055614    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:36:29.055687 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.055640    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:36:29.055687 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.055649    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mfq7\" (UniqueName: \"kubernetes.io/projected/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a-kube-api-access-2mfq7\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:36:29.796929 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.796874    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" containerID="6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c" exitCode=0
Jul 16 22:36:29.797364 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.796955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2" event={"ID":"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a","Type":"ContainerDied","Data":"6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c"}
Jul 16 22:36:29.797364 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.797020    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2" event={"ID":"2ce925cf-a3ef-4d24-8cbb-59177bd81e7a","Type":"ContainerDied","Data":"8926cf42aa95751036974c85267e8e9b50b28acd3071aaa050943807ab47c53a"}
Jul 16 22:36:29.797364 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.797034    2564 scope.go:117] "RemoveContainer" containerID="6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c"
Jul 16 22:36:29.797364 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.796964    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"
Jul 16 22:36:29.806535 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.806514    2564 scope.go:117] "RemoveContainer" containerID="6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c"
Jul 16 22:36:29.806817 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:36:29.806798    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c\": container with ID starting with 6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c not found: ID does not exist" containerID="6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c"
Jul 16 22:36:29.806868 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.806824    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c"} err="failed to get container status \"6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c\": rpc error: code = NotFound desc = could not find container \"6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c\": container with ID starting with 6468f75610b23f748caa2714f03f5c34983cb15c110dc19909b027ed574ef34c not found: ID does not exist"
Jul 16 22:36:29.819015 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.818992    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"]
Jul 16 22:36:29.826048 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:29.826024    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fb6c08-7d5f458485-dzdv2"]
Jul 16 22:36:30.126106 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:30.126027    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" path="/var/lib/kubelet/pods/2ce925cf-a3ef-4d24-8cbb-59177bd81e7a/volumes"
Jul 16 22:36:33.028282 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:33.028240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:40.607015 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:40.606982    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:40.799181 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:40.799147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:43.878927 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:43.878874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:44.106042 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:44.106009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:44.856638 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:44.856554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:44.952674 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:44.952640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:46.615286 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:46.615255    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:46.714144 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:46.714114    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:54.937621 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:54.937580    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"]
Jul 16 22:36:54.938115 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:54.937826    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn" podUID="765a4764-08ea-405d-9c5b-d6190ad63949" containerName="maas-api" containerID="cri-o://9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb" gracePeriod=30
Jul 16 22:36:55.024420 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.024383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:55.187268 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.187244    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:36:55.270698 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.270670    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx4fw\" (UniqueName: \"kubernetes.io/projected/765a4764-08ea-405d-9c5b-d6190ad63949-kube-api-access-rx4fw\") pod \"765a4764-08ea-405d-9c5b-d6190ad63949\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") "
Jul 16 22:36:55.270873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.270717    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls\") pod \"765a4764-08ea-405d-9c5b-d6190ad63949\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") "
Jul 16 22:36:55.270873 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.270798    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/765a4764-08ea-405d-9c5b-d6190ad63949-openshift-service-ca\") pod \"765a4764-08ea-405d-9c5b-d6190ad63949\" (UID: \"765a4764-08ea-405d-9c5b-d6190ad63949\") "
Jul 16 22:36:55.271221 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.271196    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/765a4764-08ea-405d-9c5b-d6190ad63949-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "765a4764-08ea-405d-9c5b-d6190ad63949" (UID: "765a4764-08ea-405d-9c5b-d6190ad63949"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:36:55.272976 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.272940    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/765a4764-08ea-405d-9c5b-d6190ad63949-kube-api-access-rx4fw" (OuterVolumeSpecName: "kube-api-access-rx4fw") pod "765a4764-08ea-405d-9c5b-d6190ad63949" (UID: "765a4764-08ea-405d-9c5b-d6190ad63949"). InnerVolumeSpecName "kube-api-access-rx4fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:36:55.272976 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.272949    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "765a4764-08ea-405d-9c5b-d6190ad63949" (UID: "765a4764-08ea-405d-9c5b-d6190ad63949"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:36:55.372352 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.372317    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/765a4764-08ea-405d-9c5b-d6190ad63949-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:36:55.372352 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.372343    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx4fw\" (UniqueName: \"kubernetes.io/projected/765a4764-08ea-405d-9c5b-d6190ad63949-kube-api-access-rx4fw\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:36:55.372352 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.372353    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/765a4764-08ea-405d-9c5b-d6190ad63949-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:36:55.936187 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.936151    2564 generic.go:356] "Generic (PLEG): container finished" podID="765a4764-08ea-405d-9c5b-d6190ad63949" containerID="9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb" exitCode=0
Jul 16 22:36:55.936390 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.936201    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"
Jul 16 22:36:55.936390 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.936236    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn" event={"ID":"765a4764-08ea-405d-9c5b-d6190ad63949","Type":"ContainerDied","Data":"9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb"}
Jul 16 22:36:55.936390 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.936300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn" event={"ID":"765a4764-08ea-405d-9c5b-d6190ad63949","Type":"ContainerDied","Data":"edb98dbebd12317bf780680cc4e2dbb5d1a5afe8bb99def13cb0624cc82d6aa9"}
Jul 16 22:36:55.936390 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.936327    2564 scope.go:117] "RemoveContainer" containerID="9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb"
Jul 16 22:36:55.947845 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.947626    2564 scope.go:117] "RemoveContainer" containerID="9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb"
Jul 16 22:36:55.948209 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:36:55.947889    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb\": container with ID starting with 9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb not found: ID does not exist" containerID="9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb"
Jul 16 22:36:55.948209 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.947938    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb"} err="failed to get container status \"9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb\": rpc error: code = NotFound desc = could not find container \"9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb\": container with ID starting with 9efd600e5b45f9bd027c67e043a8b60b2e8cacb6aa576e43ae82c6fa14360bdb not found: ID does not exist"
Jul 16 22:36:55.959318 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.959288    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"]
Jul 16 22:36:55.963408 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:55.963385    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2dbd48-788559cf8f-7sthn"]
Jul 16 22:36:56.126084 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:56.126054    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="765a4764-08ea-405d-9c5b-d6190ad63949" path="/var/lib/kubelet/pods/765a4764-08ea-405d-9c5b-d6190ad63949/volumes"
Jul 16 22:36:59.361248 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.361216    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"]
Jul 16 22:36:59.361634 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.361598    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="765a4764-08ea-405d-9c5b-d6190ad63949" containerName="maas-api"
Jul 16 22:36:59.361634 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.361608    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="765a4764-08ea-405d-9c5b-d6190ad63949" containerName="maas-api"
Jul 16 22:36:59.361634 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.361629    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" containerName="maas-api"
Jul 16 22:36:59.361634 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.361635    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" containerName="maas-api"
Jul 16 22:36:59.361768 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.361693    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ce925cf-a3ef-4d24-8cbb-59177bd81e7a" containerName="maas-api"
Jul 16 22:36:59.361768 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.361704    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="765a4764-08ea-405d-9c5b-d6190ad63949" containerName="maas-api"
Jul 16 22:36:59.364829 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.364809    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"
Jul 16 22:36:59.367667 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.367645    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:36:59.376296 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.376264    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"]
Jul 16 22:36:59.378467 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.378443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:36:59.510927 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.510877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f5b198-47f0-45ab-b2c9-7f95e46bd172-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk\" (UID: \"b0f5b198-47f0-45ab-b2c9-7f95e46bd172\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"
Jul 16 22:36:59.611707 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.611625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f5b198-47f0-45ab-b2c9-7f95e46bd172-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk\" (UID: \"b0f5b198-47f0-45ab-b2c9-7f95e46bd172\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"
Jul 16 22:36:59.612350 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.612329    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f5b198-47f0-45ab-b2c9-7f95e46bd172-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk\" (UID: \"b0f5b198-47f0-45ab-b2c9-7f95e46bd172\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"
Jul 16 22:36:59.677453 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.677426    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"
Jul 16 22:36:59.832452 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.832426    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"]
Jul 16 22:36:59.835186 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:36:59.835160    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f5b198_47f0_45ab_b2c9_7f95e46bd172.slice/crio-8c44f28b5cc8354cad08e7ca56f0172cf76a6651b0225bbfee65aac11553abad WatchSource:0}: Error finding container 8c44f28b5cc8354cad08e7ca56f0172cf76a6651b0225bbfee65aac11553abad: Status 404 returned error can't find the container with id 8c44f28b5cc8354cad08e7ca56f0172cf76a6651b0225bbfee65aac11553abad
Jul 16 22:36:59.960105 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:36:59.960065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" event={"ID":"b0f5b198-47f0-45ab-b2c9-7f95e46bd172","Type":"ContainerStarted","Data":"8c44f28b5cc8354cad08e7ca56f0172cf76a6651b0225bbfee65aac11553abad"}
Jul 16 22:37:00.967485 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:00.967448    2564 generic.go:356] "Generic (PLEG): container finished" podID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerID="18fce354d5e5c352a8f98f3a05401deb5c1509ffc8d31fcf832c587bc5944f7d" exitCode=6
Jul 16 22:37:00.967877 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:00.967540    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" event={"ID":"b0f5b198-47f0-45ab-b2c9-7f95e46bd172","Type":"ContainerDied","Data":"18fce354d5e5c352a8f98f3a05401deb5c1509ffc8d31fcf832c587bc5944f7d"}
Jul 16 22:37:00.967877 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:00.967715    2564 scope.go:117] "RemoveContainer" containerID="18fce354d5e5c352a8f98f3a05401deb5c1509ffc8d31fcf832c587bc5944f7d"
Jul 16 22:37:01.974853 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:01.974815    2564 generic.go:356] "Generic (PLEG): container finished" podID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerID="36c9e7fd93dcd1443aa277ce5aa63c03ac07afecc2fce727d8214a15b93e1a8b" exitCode=6
Jul 16 22:37:01.975337 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:01.974877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" event={"ID":"b0f5b198-47f0-45ab-b2c9-7f95e46bd172","Type":"ContainerDied","Data":"36c9e7fd93dcd1443aa277ce5aa63c03ac07afecc2fce727d8214a15b93e1a8b"}
Jul 16 22:37:01.975337 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:01.974964    2564 scope.go:117] "RemoveContainer" containerID="18fce354d5e5c352a8f98f3a05401deb5c1509ffc8d31fcf832c587bc5944f7d"
Jul 16 22:37:01.975337 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:01.975234    2564 scope.go:117] "RemoveContainer" containerID="36c9e7fd93dcd1443aa277ce5aa63c03ac07afecc2fce727d8214a15b93e1a8b"
Jul 16 22:37:01.975506 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:37:01.975432    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk_odh-ai-gateway-infra(b0f5b198-47f0-45ab-b2c9-7f95e46bd172)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172"
Jul 16 22:37:02.981440 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:02.981417    2564 scope.go:117] "RemoveContainer" containerID="36c9e7fd93dcd1443aa277ce5aa63c03ac07afecc2fce727d8214a15b93e1a8b"
Jul 16 22:37:02.981816 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:37:02.981595    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk_odh-ai-gateway-infra(b0f5b198-47f0-45ab-b2c9-7f95e46bd172)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172"
Jul 16 22:37:05.712957 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:05.712898    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:05.804968 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:05.804935    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:10.078503 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:10.078466    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:10.315965 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:10.315906    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:10.684217 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:10.684186    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:11.035732 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:11.035677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:11.325473 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:11.325356    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:13.744363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:13.744326    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:13.839580 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:13.839545    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:13.898097 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:13.898065    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"]
Jul 16 22:37:13.902036 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:13.902008    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:13.904734 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:13.904711    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-14b5b4-serving-cert\""
Jul 16 22:37:13.921290 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:13.921262    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"]
Jul 16 22:37:13.959557 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:13.959516    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:14.038781 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.038701    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/318358b0-a6bf-4f0b-9236-957a369b14c1-openshift-service-ca\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.038781 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.038745    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/318358b0-a6bf-4f0b-9236-957a369b14c1-maas-api-tls\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.038781 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.038765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z62hf\" (UniqueName: \"kubernetes.io/projected/318358b0-a6bf-4f0b-9236-957a369b14c1-kube-api-access-z62hf\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.139613 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.139581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/318358b0-a6bf-4f0b-9236-957a369b14c1-maas-api-tls\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.139613 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.139612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z62hf\" (UniqueName: \"kubernetes.io/projected/318358b0-a6bf-4f0b-9236-957a369b14c1-kube-api-access-z62hf\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.139817 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.139681    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/318358b0-a6bf-4f0b-9236-957a369b14c1-openshift-service-ca\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.140305 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.140286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/318358b0-a6bf-4f0b-9236-957a369b14c1-openshift-service-ca\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.142409 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.142385    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/318358b0-a6bf-4f0b-9236-957a369b14c1-maas-api-tls\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.149252 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.149225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z62hf\" (UniqueName: \"kubernetes.io/projected/318358b0-a6bf-4f0b-9236-957a369b14c1-kube-api-access-z62hf\") pod \"maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.150023 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.149990    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:14.213353 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.213322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:14.357216 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.357190    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"]
Jul 16 22:37:14.360002 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:37:14.359965    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318358b0_a6bf_4f0b_9236_957a369b14c1.slice/crio-50109358d25d72f809683b5a1979c9c15ecd65e1413f6c8830e2acd6ec0fe7fe WatchSource:0}: Error finding container 50109358d25d72f809683b5a1979c9c15ecd65e1413f6c8830e2acd6ec0fe7fe: Status 404 returned error can't find the container with id 50109358d25d72f809683b5a1979c9c15ecd65e1413f6c8830e2acd6ec0fe7fe
Jul 16 22:37:14.361593 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.361570    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:37:14.442446 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.442413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:14.624830 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:14.624725    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:15.045747 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:15.045710    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms" event={"ID":"318358b0-a6bf-4f0b-9236-957a369b14c1","Type":"ContainerStarted","Data":"552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd"}
Jul 16 22:37:15.046151 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:15.045797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms" event={"ID":"318358b0-a6bf-4f0b-9236-957a369b14c1","Type":"ContainerStarted","Data":"50109358d25d72f809683b5a1979c9c15ecd65e1413f6c8830e2acd6ec0fe7fe"}
Jul 16 22:37:15.046151 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:15.045931    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:15.065845 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:15.065799    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms" podStartSLOduration=1.503188206 podStartE2EDuration="2.065783763s" totalImagesPullingTime="562.595557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:37:15.064400997 +0000 UTC m=+2901.453674799" watchObservedRunningTime="2026-07-16 22:37:15.065783763 +0000 UTC m=+2901.455057548"
Jul 16 22:37:16.119552 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:16.119521    2564 scope.go:117] "RemoveContainer" containerID="36c9e7fd93dcd1443aa277ce5aa63c03ac07afecc2fce727d8214a15b93e1a8b"
Jul 16 22:37:17.059343 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:17.059247    2564 generic.go:356] "Generic (PLEG): container finished" podID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerID="41abf47ac77dec69be83981eb74c783df16c88b0e077caee8ee60b8a43b3d829" exitCode=6
Jul 16 22:37:17.059343 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:17.059323    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" event={"ID":"b0f5b198-47f0-45ab-b2c9-7f95e46bd172","Type":"ContainerDied","Data":"41abf47ac77dec69be83981eb74c783df16c88b0e077caee8ee60b8a43b3d829"}
Jul 16 22:37:17.059573 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:17.059417    2564 scope.go:117] "RemoveContainer" containerID="36c9e7fd93dcd1443aa277ce5aa63c03ac07afecc2fce727d8214a15b93e1a8b"
Jul 16 22:37:17.059728 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:17.059713    2564 scope.go:117] "RemoveContainer" containerID="41abf47ac77dec69be83981eb74c783df16c88b0e077caee8ee60b8a43b3d829"
Jul 16 22:37:17.059974 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:37:17.059954    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk_odh-ai-gateway-infra(b0f5b198-47f0-45ab-b2c9-7f95e46bd172)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172"
Jul 16 22:37:18.095060 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:18.095025    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"]
Jul 16 22:37:18.239920 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:18.239884    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"
Jul 16 22:37:18.253524 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:18.253490    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:18.375848 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:18.375771    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f5b198-47f0-45ab-b2c9-7f95e46bd172-maas-api-service-ca\") pod \"b0f5b198-47f0-45ab-b2c9-7f95e46bd172\" (UID: \"b0f5b198-47f0-45ab-b2c9-7f95e46bd172\") "
Jul 16 22:37:18.376189 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:18.376157    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f5b198-47f0-45ab-b2c9-7f95e46bd172-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0f5b198-47f0-45ab-b2c9-7f95e46bd172" (UID: "b0f5b198-47f0-45ab-b2c9-7f95e46bd172"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:37:18.476725 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:18.476688    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0f5b198-47f0-45ab-b2c9-7f95e46bd172-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:37:18.734832 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:18.734802    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:19.072399 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:19.072320    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk" event={"ID":"b0f5b198-47f0-45ab-b2c9-7f95e46bd172","Type":"ContainerDied","Data":"8c44f28b5cc8354cad08e7ca56f0172cf76a6651b0225bbfee65aac11553abad"}
Jul 16 22:37:19.072399 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:19.072356    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"
Jul 16 22:37:19.072399 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:19.072384    2564 scope.go:117] "RemoveContainer" containerID="41abf47ac77dec69be83981eb74c783df16c88b0e077caee8ee60b8a43b3d829"
Jul 16 22:37:19.104550 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:19.104514    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"]
Jul 16 22:37:19.116103 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:19.115971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:19.119482 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:19.119459    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-xzfgk"]
Jul 16 22:37:19.204241 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:19.204202    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:20.132106 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:20.127520    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" path="/var/lib/kubelet/pods/b0f5b198-47f0-45ab-b2c9-7f95e46bd172/volumes"
Jul 16 22:37:21.058145 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.058120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:21.829270 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.829236    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:21.935418 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.935395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:21.952094 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952068    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"]
Jul 16 22:37:21.952495 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952482    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:37:21.952544 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952498    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:37:21.952544 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952509    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:37:21.952544 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952516    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:37:21.952635 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952613    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:37:21.952635 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952626    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:37:21.952694 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.952637    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:37:21.957131 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.957108    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:21.960062 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.960045    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ea0330-serving-cert\""
Jul 16 22:37:21.987988 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:21.987965    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"]
Jul 16 22:37:22.078834 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.078801    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:22.109867 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.109779    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2k6x\" (UniqueName: \"kubernetes.io/projected/278b609f-ad8a-40e9-8467-a4562e7fff07-kube-api-access-v2k6x\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.109867 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.109851    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/278b609f-ad8a-40e9-8467-a4562e7fff07-maas-api-tls\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.110090 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.109893    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/278b609f-ad8a-40e9-8467-a4562e7fff07-openshift-service-ca\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.203435 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.203401    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:22.210585 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.210562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/278b609f-ad8a-40e9-8467-a4562e7fff07-maas-api-tls\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.210693 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.210599    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/278b609f-ad8a-40e9-8467-a4562e7fff07-openshift-service-ca\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.210693 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.210661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2k6x\" (UniqueName: \"kubernetes.io/projected/278b609f-ad8a-40e9-8467-a4562e7fff07-kube-api-access-v2k6x\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.211370 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.211352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/278b609f-ad8a-40e9-8467-a4562e7fff07-openshift-service-ca\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.213124 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.213101    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/278b609f-ad8a-40e9-8467-a4562e7fff07-maas-api-tls\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.221926 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.221895    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2k6x\" (UniqueName: \"kubernetes.io/projected/278b609f-ad8a-40e9-8467-a4562e7fff07-kube-api-access-v2k6x\") pod \"maas-api-e2e-api-b-ea0330-74cd97c948-z68tj\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.266982 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.266959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:22.414078 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.414048    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"]
Jul 16 22:37:22.648922 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.648874    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:22.752523 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:22.752493    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:23.094713 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:23.094614    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj" event={"ID":"278b609f-ad8a-40e9-8467-a4562e7fff07","Type":"ContainerStarted","Data":"cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0"}
Jul 16 22:37:23.094713 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:23.094674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj" event={"ID":"278b609f-ad8a-40e9-8467-a4562e7fff07","Type":"ContainerStarted","Data":"f3803fafd745d321bfbc3e2ae3dee83bafaf2d7b6b253458c1b4226acfc4b18f"}
Jul 16 22:37:23.095271 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:23.094781    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:23.118095 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:23.118053    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj" podStartSLOduration=1.728060342 podStartE2EDuration="2.118038362s" totalImagesPullingTime="389.97802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:37:23.11746076 +0000 UTC m=+2909.506734546" watchObservedRunningTime="2026-07-16 22:37:23.118038362 +0000 UTC m=+2909.507312148"
Jul 16 22:37:26.284997 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:26.284962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:29.107204 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:29.107178    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:37:35.323884 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:35.323847    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:35.424652 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:35.424620    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:44.056500 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:44.056461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:44.157094 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:44.157052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:44.854786 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:44.854749    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:44.955703 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:44.955670    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:45.121157 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.121073    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"]
Jul 16 22:37:45.121827 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.121796    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms" podUID="318358b0-a6bf-4f0b-9236-957a369b14c1" containerName="maas-api" containerID="cri-o://552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd" gracePeriod=30
Jul 16 22:37:45.146517 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.146492    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:45.387101 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.387044    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:45.511156 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.511128    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/318358b0-a6bf-4f0b-9236-957a369b14c1-openshift-service-ca\") pod \"318358b0-a6bf-4f0b-9236-957a369b14c1\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") "
Jul 16 22:37:45.511333 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.511196    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/318358b0-a6bf-4f0b-9236-957a369b14c1-maas-api-tls\") pod \"318358b0-a6bf-4f0b-9236-957a369b14c1\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") "
Jul 16 22:37:45.511333 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.511214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z62hf\" (UniqueName: \"kubernetes.io/projected/318358b0-a6bf-4f0b-9236-957a369b14c1-kube-api-access-z62hf\") pod \"318358b0-a6bf-4f0b-9236-957a369b14c1\" (UID: \"318358b0-a6bf-4f0b-9236-957a369b14c1\") "
Jul 16 22:37:45.511468 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.511442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318358b0-a6bf-4f0b-9236-957a369b14c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "318358b0-a6bf-4f0b-9236-957a369b14c1" (UID: "318358b0-a6bf-4f0b-9236-957a369b14c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:37:45.513443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.513413    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318358b0-a6bf-4f0b-9236-957a369b14c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "318358b0-a6bf-4f0b-9236-957a369b14c1" (UID: "318358b0-a6bf-4f0b-9236-957a369b14c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:37:45.513443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.513426    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/318358b0-a6bf-4f0b-9236-957a369b14c1-kube-api-access-z62hf" (OuterVolumeSpecName: "kube-api-access-z62hf") pod "318358b0-a6bf-4f0b-9236-957a369b14c1" (UID: "318358b0-a6bf-4f0b-9236-957a369b14c1"). InnerVolumeSpecName "kube-api-access-z62hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:37:45.612340 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.612308    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/318358b0-a6bf-4f0b-9236-957a369b14c1-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:37:45.612340 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.612334    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/318358b0-a6bf-4f0b-9236-957a369b14c1-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:37:45.612340 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:45.612343    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z62hf\" (UniqueName: \"kubernetes.io/projected/318358b0-a6bf-4f0b-9236-957a369b14c1-kube-api-access-z62hf\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:37:46.210360 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.210327    2564 generic.go:356] "Generic (PLEG): container finished" podID="318358b0-a6bf-4f0b-9236-957a369b14c1" containerID="552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd" exitCode=0
Jul 16 22:37:46.210780 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.210381    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"
Jul 16 22:37:46.210780 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.210409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms" event={"ID":"318358b0-a6bf-4f0b-9236-957a369b14c1","Type":"ContainerDied","Data":"552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd"}
Jul 16 22:37:46.210780 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.210461    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms" event={"ID":"318358b0-a6bf-4f0b-9236-957a369b14c1","Type":"ContainerDied","Data":"50109358d25d72f809683b5a1979c9c15ecd65e1413f6c8830e2acd6ec0fe7fe"}
Jul 16 22:37:46.210780 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.210482    2564 scope.go:117] "RemoveContainer" containerID="552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd"
Jul 16 22:37:46.222208 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.222189    2564 scope.go:117] "RemoveContainer" containerID="552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd"
Jul 16 22:37:46.222474 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:37:46.222456    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd\": container with ID starting with 552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd not found: ID does not exist" containerID="552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd"
Jul 16 22:37:46.222559 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.222483    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd"} err="failed to get container status \"552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd\": rpc error: code = NotFound desc = could not find container \"552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd\": container with ID starting with 552424bde43d1d36819f404d51249420b1dd99455a18f484ce9501fd1aff9afd not found: ID does not exist"
Jul 16 22:37:46.230137 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.230102    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"]
Jul 16 22:37:46.235269 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:46.235248    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-14b5b4-fc4bfcb4f-2mbms"]
Jul 16 22:37:48.125459 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:48.125426    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="318358b0-a6bf-4f0b-9236-957a369b14c1" path="/var/lib/kubelet/pods/318358b0-a6bf-4f0b-9236-957a369b14c1/volumes"
Jul 16 22:37:49.626880 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:49.626849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:55.676724 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:55.676692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:55.779792 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:37:55.779761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:00.394254 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:00.394220    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:00.777585 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:00.777527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:11.241561 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.241529    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"]
Jul 16 22:38:11.241955 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.241713    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj" podUID="278b609f-ad8a-40e9-8467-a4562e7fff07" containerName="maas-api" containerID="cri-o://cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0" gracePeriod=30
Jul 16 22:38:11.263172 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.263147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:11.496665 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.496602    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:38:11.642285 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.642249    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/278b609f-ad8a-40e9-8467-a4562e7fff07-openshift-service-ca\") pod \"278b609f-ad8a-40e9-8467-a4562e7fff07\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") "
Jul 16 22:38:11.642467 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.642316    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/278b609f-ad8a-40e9-8467-a4562e7fff07-maas-api-tls\") pod \"278b609f-ad8a-40e9-8467-a4562e7fff07\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") "
Jul 16 22:38:11.642467 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.642347    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2k6x\" (UniqueName: \"kubernetes.io/projected/278b609f-ad8a-40e9-8467-a4562e7fff07-kube-api-access-v2k6x\") pod \"278b609f-ad8a-40e9-8467-a4562e7fff07\" (UID: \"278b609f-ad8a-40e9-8467-a4562e7fff07\") "
Jul 16 22:38:11.642655 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.642634    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/278b609f-ad8a-40e9-8467-a4562e7fff07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "278b609f-ad8a-40e9-8467-a4562e7fff07" (UID: "278b609f-ad8a-40e9-8467-a4562e7fff07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:38:11.644604 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.644569    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/278b609f-ad8a-40e9-8467-a4562e7fff07-kube-api-access-v2k6x" (OuterVolumeSpecName: "kube-api-access-v2k6x") pod "278b609f-ad8a-40e9-8467-a4562e7fff07" (UID: "278b609f-ad8a-40e9-8467-a4562e7fff07"). InnerVolumeSpecName "kube-api-access-v2k6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:38:11.644604 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.644584    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/278b609f-ad8a-40e9-8467-a4562e7fff07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "278b609f-ad8a-40e9-8467-a4562e7fff07" (UID: "278b609f-ad8a-40e9-8467-a4562e7fff07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:38:11.743773 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.743744    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/278b609f-ad8a-40e9-8467-a4562e7fff07-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:38:11.743773 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.743770    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/278b609f-ad8a-40e9-8467-a4562e7fff07-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:38:11.743773 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:11.743781    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2k6x\" (UniqueName: \"kubernetes.io/projected/278b609f-ad8a-40e9-8467-a4562e7fff07-kube-api-access-v2k6x\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:38:12.346835 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.346805    2564 generic.go:356] "Generic (PLEG): container finished" podID="278b609f-ad8a-40e9-8467-a4562e7fff07" containerID="cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0" exitCode=0
Jul 16 22:38:12.347202 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.346855    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"
Jul 16 22:38:12.347202 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.346894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj" event={"ID":"278b609f-ad8a-40e9-8467-a4562e7fff07","Type":"ContainerDied","Data":"cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0"}
Jul 16 22:38:12.347202 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.346965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj" event={"ID":"278b609f-ad8a-40e9-8467-a4562e7fff07","Type":"ContainerDied","Data":"f3803fafd745d321bfbc3e2ae3dee83bafaf2d7b6b253458c1b4226acfc4b18f"}
Jul 16 22:38:12.347202 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.346998    2564 scope.go:117] "RemoveContainer" containerID="cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0"
Jul 16 22:38:12.356994 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.356959    2564 scope.go:117] "RemoveContainer" containerID="cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0"
Jul 16 22:38:12.357421 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:38:12.357400    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0\": container with ID starting with cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0 not found: ID does not exist" containerID="cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0"
Jul 16 22:38:12.357502 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.357428    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0"} err="failed to get container status \"cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0\": rpc error: code = NotFound desc = could not find container \"cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0\": container with ID starting with cd2839d78d379dee16b66bbf7cf4bb3ee7b8d4a45b541096b27471ad8edca2e0 not found: ID does not exist"
Jul 16 22:38:12.365890 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.365855    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"]
Jul 16 22:38:12.372757 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:12.372733    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea0330-74cd97c948-z68tj"]
Jul 16 22:38:14.125585 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:14.125549    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="278b609f-ad8a-40e9-8467-a4562e7fff07" path="/var/lib/kubelet/pods/278b609f-ad8a-40e9-8467-a4562e7fff07/volumes"
Jul 16 22:38:15.857286 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:15.857204    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:21.926161 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:21.926120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:21.996368 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:21.996332    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:26.569655 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:26.569622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:26.830245 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:26.830167    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:27.189303 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:27.189274    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:27.548191 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:27.548114    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:27.700052 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:27.700020    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:30.236535 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.236490    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:30.392329 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.392294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:30.515890 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.515817    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"]
Jul 16 22:38:30.516258 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516233    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318358b0-a6bf-4f0b-9236-957a369b14c1" containerName="maas-api"
Jul 16 22:38:30.516258 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516251    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="318358b0-a6bf-4f0b-9236-957a369b14c1" containerName="maas-api"
Jul 16 22:38:30.516258 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516264    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="278b609f-ad8a-40e9-8467-a4562e7fff07" containerName="maas-api"
Jul 16 22:38:30.516443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516270    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="278b609f-ad8a-40e9-8467-a4562e7fff07" containerName="maas-api"
Jul 16 22:38:30.516443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516283    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:38:30.516443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516289    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f5b198-47f0-45ab-b2c9-7f95e46bd172" containerName="revoke-keys"
Jul 16 22:38:30.516443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516355    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="278b609f-ad8a-40e9-8467-a4562e7fff07" containerName="maas-api"
Jul 16 22:38:30.516443 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.516364    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="318358b0-a6bf-4f0b-9236-957a369b14c1" containerName="maas-api"
Jul 16 22:38:30.520973 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.520955    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.524456 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.524429    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-bb55e2-serving-cert\""
Jul 16 22:38:30.567580 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.567551    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"]
Jul 16 22:38:30.601772 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.601737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.601963 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.601842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdxg7\" (UniqueName: \"kubernetes.io/projected/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-kube-api-access-qdxg7\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.601963 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.601878    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.603926 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.603891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:30.702877 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.702843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdxg7\" (UniqueName: \"kubernetes.io/projected/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-kube-api-access-qdxg7\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.703094 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.702881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.703094 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.702938    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.703692 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.703672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.705588 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.705567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.710064 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.710030    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:30.712270 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.712251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdxg7\" (UniqueName: \"kubernetes.io/projected/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-kube-api-access-qdxg7\") pod \"maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:30.830749 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:30.830650    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:31.185567 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:31.185519    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"]
Jul 16 22:38:31.187734 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:38:31.187709    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3b33d3_2c83_4c5b_b88f_36aef415f9b4.slice/crio-98e0e14491d017e1d9e25d7f4070503b2e1f2fcc52cdee4d93e0f622d1e6e3b5 WatchSource:0}: Error finding container 98e0e14491d017e1d9e25d7f4070503b2e1f2fcc52cdee4d93e0f622d1e6e3b5: Status 404 returned error can't find the container with id 98e0e14491d017e1d9e25d7f4070503b2e1f2fcc52cdee4d93e0f622d1e6e3b5
Jul 16 22:38:31.264888 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:31.264860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:31.360095 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:31.360059    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:31.447696 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:31.447665    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2" event={"ID":"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4","Type":"ContainerStarted","Data":"98e0e14491d017e1d9e25d7f4070503b2e1f2fcc52cdee4d93e0f622d1e6e3b5"}
Jul 16 22:38:32.454553 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:32.454519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2" event={"ID":"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4","Type":"ContainerStarted","Data":"343c4c8bf201ef8aa4e83d4904defedcab9325057e7fd754b83e0946080878a0"}
Jul 16 22:38:32.455042 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:32.454693    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:32.484830 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:32.484779    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2" podStartSLOduration=2.102012692 podStartE2EDuration="2.484763853s" totalImagesPullingTime="382.751161ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:38:32.481849861 +0000 UTC m=+2978.871123659" watchObservedRunningTime="2026-07-16 22:38:32.484763853 +0000 UTC m=+2978.874037638"
Jul 16 22:38:34.654613 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:34.654573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:35.011330 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:35.011299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:35.344544 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:35.344465    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:35.685768 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:35.685688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:38.319905 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.319868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:38.432737 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.432707    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"]
Jul 16 22:38:38.436267 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.436250    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.440209 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.440185    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-dbc614-serving-cert\""
Jul 16 22:38:38.461067 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.460809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxc6\" (UniqueName: \"kubernetes.io/projected/903a52f9-b416-4820-99e2-287aa554201b-kube-api-access-9dxc6\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.461067 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.460861    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/903a52f9-b416-4820-99e2-287aa554201b-maas-api-tls\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.461067 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.460989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/903a52f9-b416-4820-99e2-287aa554201b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.462525 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.462500    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"]
Jul 16 22:38:38.466746 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.466726    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:38:38.499868 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.499837    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:38.562100 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.562073    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxc6\" (UniqueName: \"kubernetes.io/projected/903a52f9-b416-4820-99e2-287aa554201b-kube-api-access-9dxc6\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.562289 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.562116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/903a52f9-b416-4820-99e2-287aa554201b-maas-api-tls\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.562289 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.562250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/903a52f9-b416-4820-99e2-287aa554201b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.563058 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.562994    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/903a52f9-b416-4820-99e2-287aa554201b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.565120 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.565095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/903a52f9-b416-4820-99e2-287aa554201b-maas-api-tls\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.581205 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.581144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxc6\" (UniqueName: \"kubernetes.io/projected/903a52f9-b416-4820-99e2-287aa554201b-kube-api-access-9dxc6\") pod \"maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.629956 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.629929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:38.745479 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.745440    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:38.747460 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.747437    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:38.893593 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:38.893562    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"]
Jul 16 22:38:38.894734 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:38:38.894712    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod903a52f9_b416_4820_99e2_287aa554201b.slice/crio-5f2ef1b941962467e3496ff8b8c897d84aa3919a751161b6dcd6216e22c8070a WatchSource:0}: Error finding container 5f2ef1b941962467e3496ff8b8c897d84aa3919a751161b6dcd6216e22c8070a: Status 404 returned error can't find the container with id 5f2ef1b941962467e3496ff8b8c897d84aa3919a751161b6dcd6216e22c8070a
Jul 16 22:38:39.095624 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:39.095543    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:39.190606 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:39.190567    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:39.491308 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:39.491272    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk" event={"ID":"903a52f9-b416-4820-99e2-287aa554201b","Type":"ContainerStarted","Data":"ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52"}
Jul 16 22:38:39.491725 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:39.491323    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk" event={"ID":"903a52f9-b416-4820-99e2-287aa554201b","Type":"ContainerStarted","Data":"5f2ef1b941962467e3496ff8b8c897d84aa3919a751161b6dcd6216e22c8070a"}
Jul 16 22:38:39.491725 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:39.491402    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:39.515805 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:39.515761    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk" podStartSLOduration=1.12895006 podStartE2EDuration="1.515747339s" totalImagesPullingTime="386.797279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:38:39.512416045 +0000 UTC m=+2985.901689840" watchObservedRunningTime="2026-07-16 22:38:39.515747339 +0000 UTC m=+2985.905021124"
Jul 16 22:38:42.688247 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:42.688212    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:45.501736 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:45.501705    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:38:53.437439 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:53.437397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:53.534767 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:53.534734    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:38:54.300034 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:54.300004    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:38:54.305663 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:54.305643    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:38:54.307905 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:54.307885    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:38:54.313493 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:38:54.313472    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:39:00.087523 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:39:00.087488    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:00.194682 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:39:00.194648    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:00.315856 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:39:00.315824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:16.395457 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:39:16.395422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:42.863473 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:39:42.863433    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:42.963477 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:39:42.963442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:53.753642 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:39:53.753567    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:40:04.368756 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:40:04.368723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:41:59.336162 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:41:59.336127    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:00.731569 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:00.731530    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:07.296353 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.296317    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"]
Jul 16 22:42:07.300007 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.299988    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.302604 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.302585    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-bb55e2\"/\"openshift-service-ca.crt\""
Jul 16 22:42:07.303479 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.303456    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-bb55e2\"/\"sub-test-model-bb55e2-kserve-self-signed-certs\""
Jul 16 22:42:07.303611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.303542    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-bb55e2\"/\"default-dockercfg-hdb74\""
Jul 16 22:42:07.303698 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.303614    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-bb55e2\"/\"kube-root-ca.crt\""
Jul 16 22:42:07.309469 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.309443    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"]
Jul 16 22:42:07.324130 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.324104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcv5z\" (UniqueName: \"kubernetes.io/projected/22148dbf-37ea-4c77-9638-60d16ac8731c-kube-api-access-kcv5z\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.324253 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.324147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-model-cache\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.324253 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.324165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-home\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.324253 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.324189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-tmp-dir\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.324253 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.324235    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-dshm\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.324413 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.324271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22148dbf-37ea-4c77-9638-60d16ac8731c-tls-certs\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.399081 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.399040    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:07.425317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425274    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcv5z\" (UniqueName: \"kubernetes.io/projected/22148dbf-37ea-4c77-9638-60d16ac8731c-kube-api-access-kcv5z\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.425470 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-model-cache\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.425470 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-home\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.425470 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425389    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-tmp-dir\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.425470 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-dshm\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.425692 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22148dbf-37ea-4c77-9638-60d16ac8731c-tls-certs\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.425831 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425813    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-home\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.425890 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.425846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-tmp-dir\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.426197 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.426173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-model-cache\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.428111 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.428085    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-dshm\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.428747 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.428728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22148dbf-37ea-4c77-9638-60d16ac8731c-tls-certs\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.434309 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.434289    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcv5z\" (UniqueName: \"kubernetes.io/projected/22148dbf-37ea-4c77-9638-60d16ac8731c-kube-api-access-kcv5z\") pod \"sub-test-model-bb55e2-kserve-59fdf57df-tt22z\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") " pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.609754 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.609667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:07.762555 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:07.762525    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"]
Jul 16 22:42:07.764827 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:42:07.764801    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22148dbf_37ea_4c77_9638_60d16ac8731c.slice/crio-5237880376b68f1d87707f241f8561c59f6e643166d9b6ced3479acca8ea5a4a WatchSource:0}: Error finding container 5237880376b68f1d87707f241f8561c59f6e643166d9b6ced3479acca8ea5a4a: Status 404 returned error can't find the container with id 5237880376b68f1d87707f241f8561c59f6e643166d9b6ced3479acca8ea5a4a
Jul 16 22:42:08.586792 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:08.586755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z" event={"ID":"22148dbf-37ea-4c77-9638-60d16ac8731c","Type":"ContainerStarted","Data":"a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7"}
Jul 16 22:42:08.587192 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:08.586805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z" event={"ID":"22148dbf-37ea-4c77-9638-60d16ac8731c","Type":"ContainerStarted","Data":"5237880376b68f1d87707f241f8561c59f6e643166d9b6ced3479acca8ea5a4a"}
Jul 16 22:42:08.616161 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:08.616111    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z" podStartSLOduration=1.616094414 podStartE2EDuration="1.616094414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:42:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:42:08.613966654 +0000 UTC m=+3195.003240447" watchObservedRunningTime="2026-07-16 22:42:08.616094414 +0000 UTC m=+3195.005368228"
Jul 16 22:42:17.610723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:17.610687    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:17.610723 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:17.610732    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:17.624656 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:17.624623    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:17.644409 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:17.644383    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:42:26.524515 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:26.524474    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:44.860683 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:44.860605    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:44.957984 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:42:44.957954    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:20.997347 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:20.997315    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"]
Jul 16 22:43:21.856523 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:21.856490    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:21.949197 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:21.949147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:30.513543 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:30.513507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:36.017965 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.017920    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z" podUID="22148dbf-37ea-4c77-9638-60d16ac8731c" containerName="main" containerID="cri-o://a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7" gracePeriod=45
Jul 16 22:43:36.273818 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.273761    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:43:36.392989 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.392951    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-home\") pod \"22148dbf-37ea-4c77-9638-60d16ac8731c\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") "
Jul 16 22:43:36.393159 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393013    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-tmp-dir\") pod \"22148dbf-37ea-4c77-9638-60d16ac8731c\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") "
Jul 16 22:43:36.393159 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393050    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22148dbf-37ea-4c77-9638-60d16ac8731c-tls-certs\") pod \"22148dbf-37ea-4c77-9638-60d16ac8731c\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") "
Jul 16 22:43:36.393159 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393084    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-dshm\") pod \"22148dbf-37ea-4c77-9638-60d16ac8731c\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") "
Jul 16 22:43:36.393159 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393138    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcv5z\" (UniqueName: \"kubernetes.io/projected/22148dbf-37ea-4c77-9638-60d16ac8731c-kube-api-access-kcv5z\") pod \"22148dbf-37ea-4c77-9638-60d16ac8731c\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") "
Jul 16 22:43:36.393336 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393161    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-model-cache\") pod \"22148dbf-37ea-4c77-9638-60d16ac8731c\" (UID: \"22148dbf-37ea-4c77-9638-60d16ac8731c\") "
Jul 16 22:43:36.393336 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393237    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-home" (OuterVolumeSpecName: "home") pod "22148dbf-37ea-4c77-9638-60d16ac8731c" (UID: "22148dbf-37ea-4c77-9638-60d16ac8731c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:43:36.393336 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393304    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22148dbf-37ea-4c77-9638-60d16ac8731c" (UID: "22148dbf-37ea-4c77-9638-60d16ac8731c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:43:36.393486 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393469    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-model-cache" (OuterVolumeSpecName: "model-cache") pod "22148dbf-37ea-4c77-9638-60d16ac8731c" (UID: "22148dbf-37ea-4c77-9638-60d16ac8731c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:43:36.393558 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393536    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:43:36.393558 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393559    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:43:36.393717 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.393571    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:43:36.395411 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.395387    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-dshm" (OuterVolumeSpecName: "dshm") pod "22148dbf-37ea-4c77-9638-60d16ac8731c" (UID: "22148dbf-37ea-4c77-9638-60d16ac8731c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:43:36.395521 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.395503    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22148dbf-37ea-4c77-9638-60d16ac8731c-kube-api-access-kcv5z" (OuterVolumeSpecName: "kube-api-access-kcv5z") pod "22148dbf-37ea-4c77-9638-60d16ac8731c" (UID: "22148dbf-37ea-4c77-9638-60d16ac8731c"). InnerVolumeSpecName "kube-api-access-kcv5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:43:36.395584 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.395541    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22148dbf-37ea-4c77-9638-60d16ac8731c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22148dbf-37ea-4c77-9638-60d16ac8731c" (UID: "22148dbf-37ea-4c77-9638-60d16ac8731c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:43:36.494239 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.494200    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22148dbf-37ea-4c77-9638-60d16ac8731c-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:43:36.494239 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.494234    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcv5z\" (UniqueName: \"kubernetes.io/projected/22148dbf-37ea-4c77-9638-60d16ac8731c-kube-api-access-kcv5z\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:43:36.494239 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:36.494249    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22148dbf-37ea-4c77-9638-60d16ac8731c-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:43:37.049320 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.049288    2564 generic.go:356] "Generic (PLEG): container finished" podID="22148dbf-37ea-4c77-9638-60d16ac8731c" containerID="a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7" exitCode=0
Jul 16 22:43:37.049728 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.049329    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"
Jul 16 22:43:37.049728 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.049379    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z" event={"ID":"22148dbf-37ea-4c77-9638-60d16ac8731c","Type":"ContainerDied","Data":"a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7"}
Jul 16 22:43:37.049728 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.049418    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z" event={"ID":"22148dbf-37ea-4c77-9638-60d16ac8731c","Type":"ContainerDied","Data":"5237880376b68f1d87707f241f8561c59f6e643166d9b6ced3479acca8ea5a4a"}
Jul 16 22:43:37.049728 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.049435    2564 scope.go:117] "RemoveContainer" containerID="a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7"
Jul 16 22:43:37.061007 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.060990    2564 scope.go:117] "RemoveContainer" containerID="a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7"
Jul 16 22:43:37.061264 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:43:37.061247    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7\": container with ID starting with a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7 not found: ID does not exist" containerID="a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7"
Jul 16 22:43:37.061323 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.061274    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7"} err="failed to get container status \"a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7\": rpc error: code = NotFound desc = could not find container \"a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7\": container with ID starting with a3ba3f04e2126520edc4c99a180e24cdbc14c8b36f2b052a41de9fe549321cc7 not found: ID does not exist"
Jul 16 22:43:37.077380 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.077358    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"]
Jul 16 22:43:37.085532 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:37.085510    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-bb55e2/sub-test-model-bb55e2-kserve-59fdf57df-tt22z"]
Jul 16 22:43:38.126317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:38.126284    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22148dbf-37ea-4c77-9638-60d16ac8731c" path="/var/lib/kubelet/pods/22148dbf-37ea-4c77-9638-60d16ac8731c/volumes"
Jul 16 22:43:54.333289 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:54.333263    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:43:54.339034 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:54.339014    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:43:54.342518 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:54.342498    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:43:54.348192 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:54.348171    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tm8hd_a215a163-d0cc-42a2-87cb-c50e41d39cfc/ovn-acl-logging/0.log"
Jul 16 22:43:54.398393 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:54.398367    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:43:54.499647 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:43:54.499617    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:01.215595 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:44:01.215564    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:01.320875 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:44:01.320833    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:18.756176 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:44:18.756104    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:03.557513 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:03.557476    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:03.959433 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:03.959398    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"]
Jul 16 22:45:03.962131 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:03.962103    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737320-7xpwt"]
Jul 16 22:45:04.127028 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:04.126986    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a09aade2-0dc6-4d4c-ad80-0f531e91b81f" path="/var/lib/kubelet/pods/a09aade2-0dc6-4d4c-ad80-0f531e91b81f/volumes"
Jul 16 22:45:04.390268 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:04.390193    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:05.388611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.386322    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7532-140566159141168"]
Jul 16 22:45:05.388611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.387325    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22148dbf-37ea-4c77-9638-60d16ac8731c" containerName="main"
Jul 16 22:45:05.388611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.387350    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="22148dbf-37ea-4c77-9638-60d16ac8731c" containerName="main"
Jul 16 22:45:05.388611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.387548    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="22148dbf-37ea-4c77-9638-60d16ac8731c" containerName="main"
Jul 16 22:45:05.391558 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.391525    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:05.394956 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.394929    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vsndr\""
Jul 16 22:45:05.397188 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.397164    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159141168"]
Jul 16 22:45:05.491229 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.491193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llpdr\" (UniqueName: \"kubernetes.io/projected/b8cd04d8-f0d2-4708-aaf6-018c6767cfdd-kube-api-access-llpdr\") pod \"test-curl-7532-140566159141168\" (UID: \"b8cd04d8-f0d2-4708-aaf6-018c6767cfdd\") " pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:05.592432 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.592392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llpdr\" (UniqueName: \"kubernetes.io/projected/b8cd04d8-f0d2-4708-aaf6-018c6767cfdd-kube-api-access-llpdr\") pod \"test-curl-7532-140566159141168\" (UID: \"b8cd04d8-f0d2-4708-aaf6-018c6767cfdd\") " pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:05.602063 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.602034    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llpdr\" (UniqueName: \"kubernetes.io/projected/b8cd04d8-f0d2-4708-aaf6-018c6767cfdd-kube-api-access-llpdr\") pod \"test-curl-7532-140566159141168\" (UID: \"b8cd04d8-f0d2-4708-aaf6-018c6767cfdd\") " pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:05.703720 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.703688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:05.844926 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.844889    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159141168"]
Jul 16 22:45:05.847163 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:45:05.847125    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8cd04d8_f0d2_4708_aaf6_018c6767cfdd.slice/crio-b71b73e347fc133b2ac014d6254c3294d327d5ae60329d41b7dee6d0043c9da3 WatchSource:0}: Error finding container b71b73e347fc133b2ac014d6254c3294d327d5ae60329d41b7dee6d0043c9da3: Status 404 returned error can't find the container with id b71b73e347fc133b2ac014d6254c3294d327d5ae60329d41b7dee6d0043c9da3
Jul 16 22:45:05.848545 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:05.848518    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:45:06.519664 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:06.519630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159141168" event={"ID":"b8cd04d8-f0d2-4708-aaf6-018c6767cfdd","Type":"ContainerStarted","Data":"b71b73e347fc133b2ac014d6254c3294d327d5ae60329d41b7dee6d0043c9da3"}
Jul 16 22:45:07.526857 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:07.526819    2564 generic.go:356] "Generic (PLEG): container finished" podID="b8cd04d8-f0d2-4708-aaf6-018c6767cfdd" containerID="051e3115deadaf22fda2257b196e0a94dfe7cf9c4a25de669f298fc205f542d2" exitCode=0
Jul 16 22:45:07.527277 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:07.526928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159141168" event={"ID":"b8cd04d8-f0d2-4708-aaf6-018c6767cfdd","Type":"ContainerDied","Data":"051e3115deadaf22fda2257b196e0a94dfe7cf9c4a25de669f298fc205f542d2"}
Jul 16 22:45:08.672437 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.672415    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:08.708758 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.708734    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7532-140566159141168_b8cd04d8-f0d2-4708-aaf6-018c6767cfdd/test-curl-7532-140566159141168/0.log"
Jul 16 22:45:08.721967 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.721943    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llpdr\" (UniqueName: \"kubernetes.io/projected/b8cd04d8-f0d2-4708-aaf6-018c6767cfdd-kube-api-access-llpdr\") pod \"b8cd04d8-f0d2-4708-aaf6-018c6767cfdd\" (UID: \"b8cd04d8-f0d2-4708-aaf6-018c6767cfdd\") "
Jul 16 22:45:08.724199 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.724170    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8cd04d8-f0d2-4708-aaf6-018c6767cfdd-kube-api-access-llpdr" (OuterVolumeSpecName: "kube-api-access-llpdr") pod "b8cd04d8-f0d2-4708-aaf6-018c6767cfdd" (UID: "b8cd04d8-f0d2-4708-aaf6-018c6767cfdd"). InnerVolumeSpecName "kube-api-access-llpdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:45:08.741116 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.741090    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7532-140566159141168"]
Jul 16 22:45:08.745417 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.745395    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7532-140566159141168"]
Jul 16 22:45:08.823591 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.823515    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llpdr\" (UniqueName: \"kubernetes.io/projected/b8cd04d8-f0d2-4708-aaf6-018c6767cfdd-kube-api-access-llpdr\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:08.879904 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.879870    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7532-140566159141552"]
Jul 16 22:45:08.880300 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.880285    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8cd04d8-f0d2-4708-aaf6-018c6767cfdd" containerName="test-curl-7532-140566159141168"
Jul 16 22:45:08.880356 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.880301    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8cd04d8-f0d2-4708-aaf6-018c6767cfdd" containerName="test-curl-7532-140566159141168"
Jul 16 22:45:08.880399 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.880379    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8cd04d8-f0d2-4708-aaf6-018c6767cfdd" containerName="test-curl-7532-140566159141168"
Jul 16 22:45:08.883523 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.883505    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141552"
Jul 16 22:45:08.891802 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.891736    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159141552"]
Jul 16 22:45:08.924478 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:08.924444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bmzd\" (UniqueName: \"kubernetes.io/projected/77befe08-ac44-4370-8879-c6c01ab8f577-kube-api-access-4bmzd\") pod \"test-curl-7532-140566159141552\" (UID: \"77befe08-ac44-4370-8879-c6c01ab8f577\") " pod="opendatahub/test-curl-7532-140566159141552"
Jul 16 22:45:09.025668 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:09.025636    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bmzd\" (UniqueName: \"kubernetes.io/projected/77befe08-ac44-4370-8879-c6c01ab8f577-kube-api-access-4bmzd\") pod \"test-curl-7532-140566159141552\" (UID: \"77befe08-ac44-4370-8879-c6c01ab8f577\") " pod="opendatahub/test-curl-7532-140566159141552"
Jul 16 22:45:09.034983 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:09.034952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bmzd\" (UniqueName: \"kubernetes.io/projected/77befe08-ac44-4370-8879-c6c01ab8f577-kube-api-access-4bmzd\") pod \"test-curl-7532-140566159141552\" (UID: \"77befe08-ac44-4370-8879-c6c01ab8f577\") " pod="opendatahub/test-curl-7532-140566159141552"
Jul 16 22:45:09.196360 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:09.196331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141552"
Jul 16 22:45:09.339720 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:09.339700    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159141552"]
Jul 16 22:45:09.342155 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:45:09.342130    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77befe08_ac44_4370_8879_c6c01ab8f577.slice/crio-1b1d52a4b877e04ab9e7ef192f275e3777838b2b231685c9c1a0977e45a3d271 WatchSource:0}: Error finding container 1b1d52a4b877e04ab9e7ef192f275e3777838b2b231685c9c1a0977e45a3d271: Status 404 returned error can't find the container with id 1b1d52a4b877e04ab9e7ef192f275e3777838b2b231685c9c1a0977e45a3d271
Jul 16 22:45:09.538041 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:09.537953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159141552" event={"ID":"77befe08-ac44-4370-8879-c6c01ab8f577","Type":"ContainerStarted","Data":"1b1d52a4b877e04ab9e7ef192f275e3777838b2b231685c9c1a0977e45a3d271"}
Jul 16 22:45:09.539293 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:09.539277    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b71b73e347fc133b2ac014d6254c3294d327d5ae60329d41b7dee6d0043c9da3"
Jul 16 22:45:09.539293 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:09.539278    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:09.550164 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:45:09.550134    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7532-140566159141168\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="b8cd04d8-f0d2-4708-aaf6-018c6767cfdd" pod="opendatahub/test-curl-7532-140566159141168"
Jul 16 22:45:10.126597 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:10.126564    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8cd04d8-f0d2-4708-aaf6-018c6767cfdd" path="/var/lib/kubelet/pods/b8cd04d8-f0d2-4708-aaf6-018c6767cfdd/volumes"
Jul 16 22:45:10.545321 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:10.545289    2564 generic.go:356] "Generic (PLEG): container finished" podID="77befe08-ac44-4370-8879-c6c01ab8f577" containerID="c9a0f3e480396498c234380d2701624a7aefbdc62ff005007b1543dc8a66453d" exitCode=0
Jul 16 22:45:10.545496 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:10.545322    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159141552" event={"ID":"77befe08-ac44-4370-8879-c6c01ab8f577","Type":"ContainerDied","Data":"c9a0f3e480396498c234380d2701624a7aefbdc62ff005007b1543dc8a66453d"}
Jul 16 22:45:11.693570 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.693544    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141552"
Jul 16 22:45:11.749263 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.749226    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bmzd\" (UniqueName: \"kubernetes.io/projected/77befe08-ac44-4370-8879-c6c01ab8f577-kube-api-access-4bmzd\") pod \"77befe08-ac44-4370-8879-c6c01ab8f577\" (UID: \"77befe08-ac44-4370-8879-c6c01ab8f577\") "
Jul 16 22:45:11.751424 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.751390    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77befe08-ac44-4370-8879-c6c01ab8f577-kube-api-access-4bmzd" (OuterVolumeSpecName: "kube-api-access-4bmzd") pod "77befe08-ac44-4370-8879-c6c01ab8f577" (UID: "77befe08-ac44-4370-8879-c6c01ab8f577"). InnerVolumeSpecName "kube-api-access-4bmzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:45:11.755633 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.755610    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7532-140566159141552_77befe08-ac44-4370-8879-c6c01ab8f577/test-curl-7532-140566159141552/0.log"
Jul 16 22:45:11.801966 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.801843    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7532-140566159141552"]
Jul 16 22:45:11.810423 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.810391    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7532-140566159141552"]
Jul 16 22:45:11.850954 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.850905    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bmzd\" (UniqueName: \"kubernetes.io/projected/77befe08-ac44-4370-8879-c6c01ab8f577-kube-api-access-4bmzd\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:11.986948 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.986894    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:11.987317 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.987304    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77befe08-ac44-4370-8879-c6c01ab8f577" containerName="test-curl-7532-140566159141552"
Jul 16 22:45:11.987379 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.987318    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77befe08-ac44-4370-8879-c6c01ab8f577" containerName="test-curl-7532-140566159141552"
Jul 16 22:45:11.987416 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.987394    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77befe08-ac44-4370-8879-c6c01ab8f577" containerName="test-curl-7532-140566159141552"
Jul 16 22:45:11.990519 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:11.990497    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:12.001899 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.001871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:12.052613 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.052535    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc2lq\" (UniqueName: \"kubernetes.io/projected/55eb8a84-b282-41d6-902b-c411602a12e9-kube-api-access-lc2lq\") pod \"test-curl-7532-140566159054144\" (UID: \"55eb8a84-b282-41d6-902b-c411602a12e9\") " pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:12.127207 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.127174    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77befe08-ac44-4370-8879-c6c01ab8f577" path="/var/lib/kubelet/pods/77befe08-ac44-4370-8879-c6c01ab8f577/volumes"
Jul 16 22:45:12.153620 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.153589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc2lq\" (UniqueName: \"kubernetes.io/projected/55eb8a84-b282-41d6-902b-c411602a12e9-kube-api-access-lc2lq\") pod \"test-curl-7532-140566159054144\" (UID: \"55eb8a84-b282-41d6-902b-c411602a12e9\") " pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:12.162871 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.162846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc2lq\" (UniqueName: \"kubernetes.io/projected/55eb8a84-b282-41d6-902b-c411602a12e9-kube-api-access-lc2lq\") pod \"test-curl-7532-140566159054144\" (UID: \"55eb8a84-b282-41d6-902b-c411602a12e9\") " pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:12.299958 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.299904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:12.442273 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.442149    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:12.444487 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:45:12.444448    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55eb8a84_b282_41d6_902b_c411602a12e9.slice/crio-06ec93eeec336a27262c95b91fd14ebe804548db93e46e20e76ae10d4eeaef68 WatchSource:0}: Error finding container 06ec93eeec336a27262c95b91fd14ebe804548db93e46e20e76ae10d4eeaef68: Status 404 returned error can't find the container with id 06ec93eeec336a27262c95b91fd14ebe804548db93e46e20e76ae10d4eeaef68
Jul 16 22:45:12.557853 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.557773    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159141552"
Jul 16 22:45:12.558058 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.557778    2564 scope.go:117] "RemoveContainer" containerID="c9a0f3e480396498c234380d2701624a7aefbdc62ff005007b1543dc8a66453d"
Jul 16 22:45:12.559231 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:12.559134    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159054144" event={"ID":"55eb8a84-b282-41d6-902b-c411602a12e9","Type":"ContainerStarted","Data":"06ec93eeec336a27262c95b91fd14ebe804548db93e46e20e76ae10d4eeaef68"}
Jul 16 22:45:13.566842 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:13.566798    2564 generic.go:356] "Generic (PLEG): container finished" podID="55eb8a84-b282-41d6-902b-c411602a12e9" containerID="e024c0ff60e02802675a434f6db80524a24d1ab6ecd69a79306dea8991652193" exitCode=0
Jul 16 22:45:13.567231 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:13.566895    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159054144" event={"ID":"55eb8a84-b282-41d6-902b-c411602a12e9","Type":"ContainerDied","Data":"e024c0ff60e02802675a434f6db80524a24d1ab6ecd69a79306dea8991652193"}
Jul 16 22:45:14.721432 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.721405    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:14.746968 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.746943    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7532-140566159054144_55eb8a84-b282-41d6-902b-c411602a12e9/test-curl-7532-140566159054144/0.log"
Jul 16 22:45:14.773928 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.773881    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:14.776101 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.776085    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc2lq\" (UniqueName: \"kubernetes.io/projected/55eb8a84-b282-41d6-902b-c411602a12e9-kube-api-access-lc2lq\") pod \"55eb8a84-b282-41d6-902b-c411602a12e9\" (UID: \"55eb8a84-b282-41d6-902b-c411602a12e9\") "
Jul 16 22:45:14.779649 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.778667    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55eb8a84-b282-41d6-902b-c411602a12e9-kube-api-access-lc2lq" (OuterVolumeSpecName: "kube-api-access-lc2lq") pod "55eb8a84-b282-41d6-902b-c411602a12e9" (UID: "55eb8a84-b282-41d6-902b-c411602a12e9"). InnerVolumeSpecName "kube-api-access-lc2lq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:45:14.789448 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.789416    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:14.877635 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.877538    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc2lq\" (UniqueName: \"kubernetes.io/projected/55eb8a84-b282-41d6-902b-c411602a12e9-kube-api-access-lc2lq\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:14.905666 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.905629    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:14.906095 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.906082    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55eb8a84-b282-41d6-902b-c411602a12e9" containerName="test-curl-7532-140566159054144"
Jul 16 22:45:14.906154 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.906097    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55eb8a84-b282-41d6-902b-c411602a12e9" containerName="test-curl-7532-140566159054144"
Jul 16 22:45:14.906190 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.906176    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="55eb8a84-b282-41d6-902b-c411602a12e9" containerName="test-curl-7532-140566159054144"
Jul 16 22:45:14.909151 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.909133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:14.916630 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.916503    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:14.978393 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:14.978353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npc7g\" (UniqueName: \"kubernetes.io/projected/9b5d0384-79b3-4e68-bbc2-7bfc5a90c894-kube-api-access-npc7g\") pod \"test-curl-7532-140566159054144\" (UID: \"9b5d0384-79b3-4e68-bbc2-7bfc5a90c894\") " pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:15.079607 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.079563    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npc7g\" (UniqueName: \"kubernetes.io/projected/9b5d0384-79b3-4e68-bbc2-7bfc5a90c894-kube-api-access-npc7g\") pod \"test-curl-7532-140566159054144\" (UID: \"9b5d0384-79b3-4e68-bbc2-7bfc5a90c894\") " pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:15.088978 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.088935    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npc7g\" (UniqueName: \"kubernetes.io/projected/9b5d0384-79b3-4e68-bbc2-7bfc5a90c894-kube-api-access-npc7g\") pod \"test-curl-7532-140566159054144\" (UID: \"9b5d0384-79b3-4e68-bbc2-7bfc5a90c894\") " pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:15.219037 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.218990    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:15.366780 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.366746    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:15.369933 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:45:15.369882    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b5d0384_79b3_4e68_bbc2_7bfc5a90c894.slice/crio-4987927ae43c252aa5fdb2b973209270006b0dea9d4e2f70cb962c18ff78938b WatchSource:0}: Error finding container 4987927ae43c252aa5fdb2b973209270006b0dea9d4e2f70cb962c18ff78938b: Status 404 returned error can't find the container with id 4987927ae43c252aa5fdb2b973209270006b0dea9d4e2f70cb962c18ff78938b
Jul 16 22:45:15.582279 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.582188    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:15.582279 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.582211    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06ec93eeec336a27262c95b91fd14ebe804548db93e46e20e76ae10d4eeaef68"
Jul 16 22:45:15.583504 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.583470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159054144" event={"ID":"9b5d0384-79b3-4e68-bbc2-7bfc5a90c894","Type":"ContainerStarted","Data":"4987927ae43c252aa5fdb2b973209270006b0dea9d4e2f70cb962c18ff78938b"}
Jul 16 22:45:15.594884 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:15.594852    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7532-140566159054144" oldPodUID="55eb8a84-b282-41d6-902b-c411602a12e9" podUID="9b5d0384-79b3-4e68-bbc2-7bfc5a90c894"
Jul 16 22:45:16.127065 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:16.127027    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55eb8a84-b282-41d6-902b-c411602a12e9" path="/var/lib/kubelet/pods/55eb8a84-b282-41d6-902b-c411602a12e9/volumes"
Jul 16 22:45:16.590639 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:16.590601    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b5d0384-79b3-4e68-bbc2-7bfc5a90c894" containerID="ebbad6d7ee629bdece130ccd71d365bbaec7d43f2f27366366479ff7d349c347" exitCode=0
Jul 16 22:45:16.590832 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:16.590683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159054144" event={"ID":"9b5d0384-79b3-4e68-bbc2-7bfc5a90c894","Type":"ContainerDied","Data":"ebbad6d7ee629bdece130ccd71d365bbaec7d43f2f27366366479ff7d349c347"}
Jul 16 22:45:17.456019 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.455987    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:17.554520 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.554490    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:17.736976 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.736946    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:17.758603 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.758580    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7532-140566159054144_9b5d0384-79b3-4e68-bbc2-7bfc5a90c894/test-curl-7532-140566159054144/0.log"
Jul 16 22:45:17.786531 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.786494    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:17.793271 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.793242    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7532-140566159054144"]
Jul 16 22:45:17.802532 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.802512    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npc7g\" (UniqueName: \"kubernetes.io/projected/9b5d0384-79b3-4e68-bbc2-7bfc5a90c894-kube-api-access-npc7g\") pod \"9b5d0384-79b3-4e68-bbc2-7bfc5a90c894\" (UID: \"9b5d0384-79b3-4e68-bbc2-7bfc5a90c894\") "
Jul 16 22:45:17.804766 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.804744    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b5d0384-79b3-4e68-bbc2-7bfc5a90c894-kube-api-access-npc7g" (OuterVolumeSpecName: "kube-api-access-npc7g") pod "9b5d0384-79b3-4e68-bbc2-7bfc5a90c894" (UID: "9b5d0384-79b3-4e68-bbc2-7bfc5a90c894"). InnerVolumeSpecName "kube-api-access-npc7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:45:17.903772 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.903736    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npc7g\" (UniqueName: \"kubernetes.io/projected/9b5d0384-79b3-4e68-bbc2-7bfc5a90c894-kube-api-access-npc7g\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:17.919600 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.919560    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:17.920002 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.919987    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b5d0384-79b3-4e68-bbc2-7bfc5a90c894" containerName="test-curl-7532-140566159054144"
Jul 16 22:45:17.920058 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.920004    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b5d0384-79b3-4e68-bbc2-7bfc5a90c894" containerName="test-curl-7532-140566159054144"
Jul 16 22:45:17.920091 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.920077    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b5d0384-79b3-4e68-bbc2-7bfc5a90c894" containerName="test-curl-7532-140566159054144"
Jul 16 22:45:17.923209 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.923108    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:17.934786 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:17.934758    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:18.005017 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.004890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqbvh\" (UniqueName: \"kubernetes.io/projected/755a7398-c19b-4ebe-83b4-8d9acc738bee-kube-api-access-zqbvh\") pod \"test-curl-7532-140566159055152\" (UID: \"755a7398-c19b-4ebe-83b4-8d9acc738bee\") " pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:18.105678 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.105636    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbvh\" (UniqueName: \"kubernetes.io/projected/755a7398-c19b-4ebe-83b4-8d9acc738bee-kube-api-access-zqbvh\") pod \"test-curl-7532-140566159055152\" (UID: \"755a7398-c19b-4ebe-83b4-8d9acc738bee\") " pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:18.121960 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.121896    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqbvh\" (UniqueName: \"kubernetes.io/projected/755a7398-c19b-4ebe-83b4-8d9acc738bee-kube-api-access-zqbvh\") pod \"test-curl-7532-140566159055152\" (UID: \"755a7398-c19b-4ebe-83b4-8d9acc738bee\") " pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:18.126975 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.126944    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b5d0384-79b3-4e68-bbc2-7bfc5a90c894" path="/var/lib/kubelet/pods/9b5d0384-79b3-4e68-bbc2-7bfc5a90c894/volumes"
Jul 16 22:45:18.234681 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.234643    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:18.384821 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.384791    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:18.386312 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:45:18.386289    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod755a7398_c19b_4ebe_83b4_8d9acc738bee.slice/crio-8ab7be1d376a12212d92f7b51ea9af0c37dd8f6e15c27ffa8b71a45ce710c4a1 WatchSource:0}: Error finding container 8ab7be1d376a12212d92f7b51ea9af0c37dd8f6e15c27ffa8b71a45ce710c4a1: Status 404 returned error can't find the container with id 8ab7be1d376a12212d92f7b51ea9af0c37dd8f6e15c27ffa8b71a45ce710c4a1
Jul 16 22:45:18.602242 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.602158    2564 scope.go:117] "RemoveContainer" containerID="ebbad6d7ee629bdece130ccd71d365bbaec7d43f2f27366366479ff7d349c347"
Jul 16 22:45:18.602682 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.602165    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159054144"
Jul 16 22:45:18.603476 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:18.603378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159055152" event={"ID":"755a7398-c19b-4ebe-83b4-8d9acc738bee","Type":"ContainerStarted","Data":"8ab7be1d376a12212d92f7b51ea9af0c37dd8f6e15c27ffa8b71a45ce710c4a1"}
Jul 16 22:45:19.611215 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:19.611177    2564 generic.go:356] "Generic (PLEG): container finished" podID="755a7398-c19b-4ebe-83b4-8d9acc738bee" containerID="2931c3f26d23dfa30845f7a76ed2164b3ccf3b984ce71d43e31e525024a478af" exitCode=0
Jul 16 22:45:19.611626 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:19.611263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159055152" event={"ID":"755a7398-c19b-4ebe-83b4-8d9acc738bee","Type":"ContainerDied","Data":"2931c3f26d23dfa30845f7a76ed2164b3ccf3b984ce71d43e31e525024a478af"}
Jul 16 22:45:20.762828 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.762800    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:20.796141 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.796118    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7532-140566159055152_755a7398-c19b-4ebe-83b4-8d9acc738bee/test-curl-7532-140566159055152/0.log"
Jul 16 22:45:20.830286 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.830252    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqbvh\" (UniqueName: \"kubernetes.io/projected/755a7398-c19b-4ebe-83b4-8d9acc738bee-kube-api-access-zqbvh\") pod \"755a7398-c19b-4ebe-83b4-8d9acc738bee\" (UID: \"755a7398-c19b-4ebe-83b4-8d9acc738bee\") "
Jul 16 22:45:20.832763 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.832734    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755a7398-c19b-4ebe-83b4-8d9acc738bee-kube-api-access-zqbvh" (OuterVolumeSpecName: "kube-api-access-zqbvh") pod "755a7398-c19b-4ebe-83b4-8d9acc738bee" (UID: "755a7398-c19b-4ebe-83b4-8d9acc738bee"). InnerVolumeSpecName "kube-api-access-zqbvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:45:20.841059 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.841031    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:20.842923 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.842893    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:20.931698 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.931617    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqbvh\" (UniqueName: \"kubernetes.io/projected/755a7398-c19b-4ebe-83b4-8d9acc738bee-kube-api-access-zqbvh\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:20.971259 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.971226    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:20.971624 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.971612    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755a7398-c19b-4ebe-83b4-8d9acc738bee" containerName="test-curl-7532-140566159055152"
Jul 16 22:45:20.971663 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.971626    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="755a7398-c19b-4ebe-83b4-8d9acc738bee" containerName="test-curl-7532-140566159055152"
Jul 16 22:45:20.971725 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.971716    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="755a7398-c19b-4ebe-83b4-8d9acc738bee" containerName="test-curl-7532-140566159055152"
Jul 16 22:45:20.974712 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.974694    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:20.986518 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:20.986495    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:21.032355 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.032320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsjxz\" (UniqueName: \"kubernetes.io/projected/60cb32cb-6e04-492e-953f-0963b231ec7b-kube-api-access-nsjxz\") pod \"test-curl-7532-140566159055152\" (UID: \"60cb32cb-6e04-492e-953f-0963b231ec7b\") " pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:21.133105 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.133065    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsjxz\" (UniqueName: \"kubernetes.io/projected/60cb32cb-6e04-492e-953f-0963b231ec7b-kube-api-access-nsjxz\") pod \"test-curl-7532-140566159055152\" (UID: \"60cb32cb-6e04-492e-953f-0963b231ec7b\") " pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:21.147293 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.147259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsjxz\" (UniqueName: \"kubernetes.io/projected/60cb32cb-6e04-492e-953f-0963b231ec7b-kube-api-access-nsjxz\") pod \"test-curl-7532-140566159055152\" (UID: \"60cb32cb-6e04-492e-953f-0963b231ec7b\") " pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:21.283605 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.283567    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:21.423840 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.423813    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:21.425860 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:45:21.425829    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60cb32cb_6e04_492e_953f_0963b231ec7b.slice/crio-6bfdf4f6c349bfaea8b342d3fa4072f46c5c55b73902870db1338dae6863e75f WatchSource:0}: Error finding container 6bfdf4f6c349bfaea8b342d3fa4072f46c5c55b73902870db1338dae6863e75f: Status 404 returned error can't find the container with id 6bfdf4f6c349bfaea8b342d3fa4072f46c5c55b73902870db1338dae6863e75f
Jul 16 22:45:21.624379 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.624290    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:21.624379 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.624320    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ab7be1d376a12212d92f7b51ea9af0c37dd8f6e15c27ffa8b71a45ce710c4a1"
Jul 16 22:45:21.625709 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.625681    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159055152" event={"ID":"60cb32cb-6e04-492e-953f-0963b231ec7b","Type":"ContainerStarted","Data":"6bfdf4f6c349bfaea8b342d3fa4072f46c5c55b73902870db1338dae6863e75f"}
Jul 16 22:45:21.640347 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:21.640321    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7532-140566159055152" oldPodUID="755a7398-c19b-4ebe-83b4-8d9acc738bee" podUID="60cb32cb-6e04-492e-953f-0963b231ec7b"
Jul 16 22:45:22.125766 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:22.125729    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="755a7398-c19b-4ebe-83b4-8d9acc738bee" path="/var/lib/kubelet/pods/755a7398-c19b-4ebe-83b4-8d9acc738bee/volumes"
Jul 16 22:45:22.633396 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:22.633361    2564 generic.go:356] "Generic (PLEG): container finished" podID="60cb32cb-6e04-492e-953f-0963b231ec7b" containerID="cfaacdfb2fc5e6d2a6e7db309107f7c911aa5ab5639ef910f220ac085a08028c" exitCode=0
Jul 16 22:45:22.633549 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:22.633451    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7532-140566159055152" event={"ID":"60cb32cb-6e04-492e-953f-0963b231ec7b","Type":"ContainerDied","Data":"cfaacdfb2fc5e6d2a6e7db309107f7c911aa5ab5639ef910f220ac085a08028c"}
Jul 16 22:45:23.783683 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:23.783656    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:23.809468 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:23.809433    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7532-140566159055152_60cb32cb-6e04-492e-953f-0963b231ec7b/test-curl-7532-140566159055152/0.log"
Jul 16 22:45:23.837640 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:23.837602    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:23.840110 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:23.840086    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7532-140566159055152"]
Jul 16 22:45:23.858420 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:23.858394    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsjxz\" (UniqueName: \"kubernetes.io/projected/60cb32cb-6e04-492e-953f-0963b231ec7b-kube-api-access-nsjxz\") pod \"60cb32cb-6e04-492e-953f-0963b231ec7b\" (UID: \"60cb32cb-6e04-492e-953f-0963b231ec7b\") "
Jul 16 22:45:23.860596 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:23.860570    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60cb32cb-6e04-492e-953f-0963b231ec7b-kube-api-access-nsjxz" (OuterVolumeSpecName: "kube-api-access-nsjxz") pod "60cb32cb-6e04-492e-953f-0963b231ec7b" (UID: "60cb32cb-6e04-492e-953f-0963b231ec7b"). InnerVolumeSpecName "kube-api-access-nsjxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:45:23.959105 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:23.959070    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsjxz\" (UniqueName: \"kubernetes.io/projected/60cb32cb-6e04-492e-953f-0963b231ec7b-kube-api-access-nsjxz\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:24.022005 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.021968    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"]
Jul 16 22:45:24.022385 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.022372    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60cb32cb-6e04-492e-953f-0963b231ec7b" containerName="test-curl-7532-140566159055152"
Jul 16 22:45:24.022431 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.022387    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60cb32cb-6e04-492e-953f-0963b231ec7b" containerName="test-curl-7532-140566159055152"
Jul 16 22:45:24.022465 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.022447    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="60cb32cb-6e04-492e-953f-0963b231ec7b" containerName="test-curl-7532-140566159055152"
Jul 16 22:45:24.025519 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.025501    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"
Jul 16 22:45:24.028560 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.028530    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:45:24.034705 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.034678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"]
Jul 16 22:45:24.126208 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.126172    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60cb32cb-6e04-492e-953f-0963b231ec7b" path="/var/lib/kubelet/pods/60cb32cb-6e04-492e-953f-0963b231ec7b/volumes"
Jul 16 22:45:24.161665 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.161629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf8509c-1f00-45aa-9f48-f92d21fe2e8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h\" (UID: \"acf8509c-1f00-45aa-9f48-f92d21fe2e8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"
Jul 16 22:45:24.262980 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.262866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf8509c-1f00-45aa-9f48-f92d21fe2e8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h\" (UID: \"acf8509c-1f00-45aa-9f48-f92d21fe2e8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"
Jul 16 22:45:24.263479 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.263458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf8509c-1f00-45aa-9f48-f92d21fe2e8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h\" (UID: \"acf8509c-1f00-45aa-9f48-f92d21fe2e8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"
Jul 16 22:45:24.337969 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.337936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"
Jul 16 22:45:24.481280 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.481168    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"]
Jul 16 22:45:24.484020 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:45:24.483991    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf8509c_1f00_45aa_9f48_f92d21fe2e8d.slice/crio-16011bf51383dc0003352f098a3a60442e20d72ab2575da8090e8f1a4c0fe0e5 WatchSource:0}: Error finding container 16011bf51383dc0003352f098a3a60442e20d72ab2575da8090e8f1a4c0fe0e5: Status 404 returned error can't find the container with id 16011bf51383dc0003352f098a3a60442e20d72ab2575da8090e8f1a4c0fe0e5
Jul 16 22:45:24.606776 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.606749    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:24.645082 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.645049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h" event={"ID":"acf8509c-1f00-45aa-9f48-f92d21fe2e8d","Type":"ContainerStarted","Data":"16011bf51383dc0003352f098a3a60442e20d72ab2575da8090e8f1a4c0fe0e5"}
Jul 16 22:45:24.646461 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.646416    2564 scope.go:117] "RemoveContainer" containerID="cfaacdfb2fc5e6d2a6e7db309107f7c911aa5ab5639ef910f220ac085a08028c"
Jul 16 22:45:24.646461 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.646449    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7532-140566159055152"
Jul 16 22:45:24.706008 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:24.705981    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:25.653689 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:25.653652    2564 generic.go:356] "Generic (PLEG): container finished" podID="acf8509c-1f00-45aa-9f48-f92d21fe2e8d" containerID="441fac39012a9a50a7f4986e769b551a22722591403dcd2fe745d16bb38f40ca" exitCode=0
Jul 16 22:45:25.654114 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:25.653748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h" event={"ID":"acf8509c-1f00-45aa-9f48-f92d21fe2e8d","Type":"ContainerDied","Data":"441fac39012a9a50a7f4986e769b551a22722591403dcd2fe745d16bb38f40ca"}
Jul 16 22:45:26.801516 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:26.801486    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"
Jul 16 22:45:26.885192 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:26.885157    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf8509c-1f00-45aa-9f48-f92d21fe2e8d-maas-api-service-ca\") pod \"acf8509c-1f00-45aa-9f48-f92d21fe2e8d\" (UID: \"acf8509c-1f00-45aa-9f48-f92d21fe2e8d\") "
Jul 16 22:45:26.885559 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:26.885540    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf8509c-1f00-45aa-9f48-f92d21fe2e8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "acf8509c-1f00-45aa-9f48-f92d21fe2e8d" (UID: "acf8509c-1f00-45aa-9f48-f92d21fe2e8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:45:26.986227 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:26.986186    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf8509c-1f00-45aa-9f48-f92d21fe2e8d-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:27.666587 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:27.666555    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h"
Jul 16 22:45:27.666745 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:27.666608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h" event={"ID":"acf8509c-1f00-45aa-9f48-f92d21fe2e8d","Type":"ContainerDied","Data":"16011bf51383dc0003352f098a3a60442e20d72ab2575da8090e8f1a4c0fe0e5"}
Jul 16 22:45:27.666745 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:27.666658    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16011bf51383dc0003352f098a3a60442e20d72ab2575da8090e8f1a4c0fe0e5"
Jul 16 22:45:34.542108 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.542067    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"]
Jul 16 22:45:34.542647 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.542332    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2" podUID="1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" containerName="maas-api" containerID="cri-o://343c4c8bf201ef8aa4e83d4904defedcab9325057e7fd754b83e0946080878a0" gracePeriod=30
Jul 16 22:45:34.572899 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.572854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:34.705471 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.705439    2564 generic.go:356] "Generic (PLEG): container finished" podID="1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" containerID="343c4c8bf201ef8aa4e83d4904defedcab9325057e7fd754b83e0946080878a0" exitCode=0
Jul 16 22:45:34.705697 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.705513    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2" event={"ID":"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4","Type":"ContainerDied","Data":"343c4c8bf201ef8aa4e83d4904defedcab9325057e7fd754b83e0946080878a0"}
Jul 16 22:45:34.799828 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.799770    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:45:34.958189 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.958157    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-maas-api-tls\") pod \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") "
Jul 16 22:45:34.958375 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.958215    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-openshift-service-ca\") pod \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") "
Jul 16 22:45:34.958375 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.958246    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdxg7\" (UniqueName: \"kubernetes.io/projected/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-kube-api-access-qdxg7\") pod \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\" (UID: \"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4\") "
Jul 16 22:45:34.958616 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.958580    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" (UID: "1a3b33d3-2c83-4c5b-b88f-36aef415f9b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:45:34.960470 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.960429    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" (UID: "1a3b33d3-2c83-4c5b-b88f-36aef415f9b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:45:34.960592 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:34.960561    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-kube-api-access-qdxg7" (OuterVolumeSpecName: "kube-api-access-qdxg7") pod "1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" (UID: "1a3b33d3-2c83-4c5b-b88f-36aef415f9b4"). InnerVolumeSpecName "kube-api-access-qdxg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:45:35.059278 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.059187    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:35.059278 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.059223    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:35.059278 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.059232    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdxg7\" (UniqueName: \"kubernetes.io/projected/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4-kube-api-access-qdxg7\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:45:35.711979 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.711941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2" event={"ID":"1a3b33d3-2c83-4c5b-b88f-36aef415f9b4","Type":"ContainerDied","Data":"98e0e14491d017e1d9e25d7f4070503b2e1f2fcc52cdee4d93e0f622d1e6e3b5"}
Jul 16 22:45:35.711979 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.711978    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"
Jul 16 22:45:35.712529 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.712015    2564 scope.go:117] "RemoveContainer" containerID="343c4c8bf201ef8aa4e83d4904defedcab9325057e7fd754b83e0946080878a0"
Jul 16 22:45:35.736387 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.736354    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"]
Jul 16 22:45:35.741454 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:35.741428    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb55e2-5569d65885-zphp2"]
Jul 16 22:45:36.133842 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:36.133751    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" path="/var/lib/kubelet/pods/1a3b33d3-2c83-4c5b-b88f-36aef415f9b4/volumes"
Jul 16 22:45:36.910846 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:36.910812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:39.156047 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:39.156009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:45.162263 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:45.162187    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:45.238090 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:45.238053    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:49.876889 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:49.876851    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:50.110272 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:50.110240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:50.912496 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:50.912462    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:51.020320 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:51.020288    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:55.134792 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:45:55.134759    2564 scope.go:117] "RemoveContainer" containerID="bf005e76034956d2ad9d6f488c24b19766fc4e15ebccbc7ef282b142dde10d75"
Jul 16 22:46:00.860854 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:00.860808    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"]
Jul 16 22:46:00.861304 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:00.861161    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk" podUID="903a52f9-b416-4820-99e2-287aa554201b" containerName="maas-api" containerID="cri-o://ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52" gracePeriod=30
Jul 16 22:46:00.893383 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:00.893342    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:01.114303 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.114220    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:46:01.179505 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.179476    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/903a52f9-b416-4820-99e2-287aa554201b-maas-api-tls\") pod \"903a52f9-b416-4820-99e2-287aa554201b\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") "
Jul 16 22:46:01.179677 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.179511    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/903a52f9-b416-4820-99e2-287aa554201b-openshift-service-ca\") pod \"903a52f9-b416-4820-99e2-287aa554201b\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") "
Jul 16 22:46:01.179677 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.179556    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dxc6\" (UniqueName: \"kubernetes.io/projected/903a52f9-b416-4820-99e2-287aa554201b-kube-api-access-9dxc6\") pod \"903a52f9-b416-4820-99e2-287aa554201b\" (UID: \"903a52f9-b416-4820-99e2-287aa554201b\") "
Jul 16 22:46:01.179954 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.179889    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/903a52f9-b416-4820-99e2-287aa554201b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "903a52f9-b416-4820-99e2-287aa554201b" (UID: "903a52f9-b416-4820-99e2-287aa554201b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:46:01.181858 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.181837    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/903a52f9-b416-4820-99e2-287aa554201b-kube-api-access-9dxc6" (OuterVolumeSpecName: "kube-api-access-9dxc6") pod "903a52f9-b416-4820-99e2-287aa554201b" (UID: "903a52f9-b416-4820-99e2-287aa554201b"). InnerVolumeSpecName "kube-api-access-9dxc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:46:01.181858 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.181846    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/903a52f9-b416-4820-99e2-287aa554201b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "903a52f9-b416-4820-99e2-287aa554201b" (UID: "903a52f9-b416-4820-99e2-287aa554201b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:46:01.280741 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.280702    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/903a52f9-b416-4820-99e2-287aa554201b-maas-api-tls\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:46:01.280741 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.280733    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/903a52f9-b416-4820-99e2-287aa554201b-openshift-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:46:01.280741 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.280745    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dxc6\" (UniqueName: \"kubernetes.io/projected/903a52f9-b416-4820-99e2-287aa554201b-kube-api-access-9dxc6\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:46:01.856428 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.856390    2564 generic.go:356] "Generic (PLEG): container finished" podID="903a52f9-b416-4820-99e2-287aa554201b" containerID="ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52" exitCode=0
Jul 16 22:46:01.856588 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.856442    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"
Jul 16 22:46:01.856588 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.856489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk" event={"ID":"903a52f9-b416-4820-99e2-287aa554201b","Type":"ContainerDied","Data":"ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52"}
Jul 16 22:46:01.856588 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.856550    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk" event={"ID":"903a52f9-b416-4820-99e2-287aa554201b","Type":"ContainerDied","Data":"5f2ef1b941962467e3496ff8b8c897d84aa3919a751161b6dcd6216e22c8070a"}
Jul 16 22:46:01.856588 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.856576    2564 scope.go:117] "RemoveContainer" containerID="ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52"
Jul 16 22:46:01.866944 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.866732    2564 scope.go:117] "RemoveContainer" containerID="ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52"
Jul 16 22:46:01.867211 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:46:01.867076    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52\": container with ID starting with ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52 not found: ID does not exist" containerID="ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52"
Jul 16 22:46:01.867211 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.867099    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52"} err="failed to get container status \"ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52\": rpc error: code = NotFound desc = could not find container \"ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52\": container with ID starting with ee867598d14974d5e857232ee49c926d5cf2c5d344ae3b44275688d590aafb52 not found: ID does not exist"
Jul 16 22:46:01.880194 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.880164    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"]
Jul 16 22:46:01.883883 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:01.883861    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-dbc614-6fc7887d65-hqhsk"]
Jul 16 22:46:02.128282 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:02.128194    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="903a52f9-b416-4820-99e2-287aa554201b" path="/var/lib/kubelet/pods/903a52f9-b416-4820-99e2-287aa554201b/volumes"
Jul 16 22:46:04.881090 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:04.881055    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:04.981884 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:04.981850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:05.337030 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:05.336995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:10.108967 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.108904    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"]
Jul 16 22:46:10.109491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109469    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf8509c-1f00-45aa-9f48-f92d21fe2e8d" containerName="revoke-keys"
Jul 16 22:46:10.109579 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109494    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf8509c-1f00-45aa-9f48-f92d21fe2e8d" containerName="revoke-keys"
Jul 16 22:46:10.109579 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109511    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="903a52f9-b416-4820-99e2-287aa554201b" containerName="maas-api"
Jul 16 22:46:10.109579 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109518    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="903a52f9-b416-4820-99e2-287aa554201b" containerName="maas-api"
Jul 16 22:46:10.109579 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109535    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" containerName="maas-api"
Jul 16 22:46:10.109579 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109544    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" containerName="maas-api"
Jul 16 22:46:10.109818 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109655    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="903a52f9-b416-4820-99e2-287aa554201b" containerName="maas-api"
Jul 16 22:46:10.109818 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109672    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a3b33d3-2c83-4c5b-b88f-36aef415f9b4" containerName="maas-api"
Jul 16 22:46:10.109818 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.109683    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf8509c-1f00-45aa-9f48-f92d21fe2e8d" containerName="revoke-keys"
Jul 16 22:46:10.112898 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.112877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"
Jul 16 22:46:10.115666 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.115648    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gknhg\""
Jul 16 22:46:10.127775 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.127750    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"]
Jul 16 22:46:10.156850 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.156812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066575ee-4dd2-4056-b416-31194f7475a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z\" (UID: \"066575ee-4dd2-4056-b416-31194f7475a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"
Jul 16 22:46:10.257756 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.257717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066575ee-4dd2-4056-b416-31194f7475a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z\" (UID: \"066575ee-4dd2-4056-b416-31194f7475a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"
Jul 16 22:46:10.258383 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.258362    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066575ee-4dd2-4056-b416-31194f7475a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z\" (UID: \"066575ee-4dd2-4056-b416-31194f7475a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"
Jul 16 22:46:10.425590 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.425491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"
Jul 16 22:46:10.565015 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.564980    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"]
Jul 16 22:46:10.566645 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:46:10.566609    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066575ee_4dd2_4056_b416_31194f7475a2.slice/crio-8f8445ea491df27e46eb691ec81b8e7e3686a8bd8b0f302100734fa332593512 WatchSource:0}: Error finding container 8f8445ea491df27e46eb691ec81b8e7e3686a8bd8b0f302100734fa332593512: Status 404 returned error can't find the container with id 8f8445ea491df27e46eb691ec81b8e7e3686a8bd8b0f302100734fa332593512
Jul 16 22:46:10.908501 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.908464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" event={"ID":"066575ee-4dd2-4056-b416-31194f7475a2","Type":"ContainerStarted","Data":"d4735b9fab338b60a8ea5cb5f57a7730ab2941ad109a73023fe6f39195ca7f2d"}
Jul 16 22:46:10.908673 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.908517    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" event={"ID":"066575ee-4dd2-4056-b416-31194f7475a2","Type":"ContainerStarted","Data":"8f8445ea491df27e46eb691ec81b8e7e3686a8bd8b0f302100734fa332593512"}
Jul 16 22:46:10.927014 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:10.926965    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" podStartSLOduration=0.9269495 podStartE2EDuration="926.9495ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:46:10.926114561 +0000 UTC m=+3437.315388347" watchObservedRunningTime="2026-07-16 22:46:10.9269495 +0000 UTC m=+3437.316223285"
Jul 16 22:46:11.592662 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:11.592635    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:11.693069 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:11.693039    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:11.915061 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:11.914964    2564 generic.go:356] "Generic (PLEG): container finished" podID="066575ee-4dd2-4056-b416-31194f7475a2" containerID="d4735b9fab338b60a8ea5cb5f57a7730ab2941ad109a73023fe6f39195ca7f2d" exitCode=6
Jul 16 22:46:11.915219 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:11.915049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" event={"ID":"066575ee-4dd2-4056-b416-31194f7475a2","Type":"ContainerDied","Data":"d4735b9fab338b60a8ea5cb5f57a7730ab2941ad109a73023fe6f39195ca7f2d"}
Jul 16 22:46:11.915263 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:11.915230    2564 scope.go:117] "RemoveContainer" containerID="d4735b9fab338b60a8ea5cb5f57a7730ab2941ad109a73023fe6f39195ca7f2d"
Jul 16 22:46:12.922011 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:12.921893    2564 generic.go:356] "Generic (PLEG): container finished" podID="066575ee-4dd2-4056-b416-31194f7475a2" containerID="7d1d1645de3c730774c10da83550bd65214c3c03826871403fd83f7da88d6822" exitCode=6
Jul 16 22:46:12.922011 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:12.921981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" event={"ID":"066575ee-4dd2-4056-b416-31194f7475a2","Type":"ContainerDied","Data":"7d1d1645de3c730774c10da83550bd65214c3c03826871403fd83f7da88d6822"}
Jul 16 22:46:12.922477 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:12.922075    2564 scope.go:117] "RemoveContainer" containerID="d4735b9fab338b60a8ea5cb5f57a7730ab2941ad109a73023fe6f39195ca7f2d"
Jul 16 22:46:12.922477 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:12.922269    2564 scope.go:117] "RemoveContainer" containerID="7d1d1645de3c730774c10da83550bd65214c3c03826871403fd83f7da88d6822"
Jul 16 22:46:12.922552 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:46:12.922487    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z_odh-ai-gateway-infra(066575ee-4dd2-4056-b416-31194f7475a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" podUID="066575ee-4dd2-4056-b416-31194f7475a2"
Jul 16 22:46:13.930481 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:13.930449    2564 scope.go:117] "RemoveContainer" containerID="7d1d1645de3c730774c10da83550bd65214c3c03826871403fd83f7da88d6822"
Jul 16 22:46:13.930960 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:46:13.930736    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z_odh-ai-gateway-infra(066575ee-4dd2-4056-b416-31194f7475a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" podUID="066575ee-4dd2-4056-b416-31194f7475a2"
Jul 16 22:46:16.092463 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:16.092422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:16.342711 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:16.342628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:16.766535 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:16.766501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9bdf84c6c-t7md7_0cc69be4-dc26-4fad-a7c4-1b46ecb29369/authorino/0.log"
Jul 16 22:46:22.549514 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:22.549483    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tlhk5_cd872a79-78f1-4c79-b884-6b2723da3867/manager/0.log"
Jul 16 22:46:22.671526 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:22.671490    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bvzc8_0653006a-640b-4264-be65-9c922d3d7f3a/manager/0.log"
Jul 16 22:46:22.797388 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:22.797356    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f4d5cdddc-wqmfp_241531e3-1b0f-492e-a104-d49454633c50/manager/0.log"
Jul 16 22:46:23.220520 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:23.220494    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k_58d9f09a-6837-4e79-b0c0-78f830eeb47d/copy-manifests/0.log"
Jul 16 22:46:23.229491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:23.229469    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k_58d9f09a-6837-4e79-b0c0-78f830eeb47d/manager/0.log"
Jul 16 22:46:23.895218 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:23.895188    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54d578c6bd-24f5x_94941e6c-ed92-4e91-876d-f2db63e8d1b2/maas-api/0.log"
Jul 16 22:46:24.024323 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:24.024288    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737335-f8v8k_70f81a2a-db21-44fe-afc3-fa55f28fc1d9/cleanup/0.log"
Jul 16 22:46:24.155076 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:24.155001    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737350-g6grt_36e4d2da-63bf-4e79-a13e-c4706abc61b5/cleanup/0.log"
Jul 16 22:46:24.753228 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:24.753199    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z_066575ee-4dd2-4056-b416-31194f7475a2/revoke-keys/1.log"
Jul 16 22:46:25.119808 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:25.119722    2564 scope.go:117] "RemoveContainer" containerID="7d1d1645de3c730774c10da83550bd65214c3c03826871403fd83f7da88d6822"
Jul 16 22:46:25.453163 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:25.453136    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-bb55e2-c9f6h_acf8509c-1f00-45aa-9f48-f92d21fe2e8d/revoke-keys/0.log"
Jul 16 22:46:25.685232 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:25.685153    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-27jcj_5d04ae75-9d74-4e95-a1f6-2d24a606d90c/postgres/0.log"
Jul 16 22:46:25.995273 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:25.995241    2564 generic.go:356] "Generic (PLEG): container finished" podID="066575ee-4dd2-4056-b416-31194f7475a2" containerID="165130e9db93462ba0876b11d8d9c1a34ad352ee77afbfe0da0ddaa45a535dd2" exitCode=6
Jul 16 22:46:25.995453 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:25.995297    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" event={"ID":"066575ee-4dd2-4056-b416-31194f7475a2","Type":"ContainerDied","Data":"165130e9db93462ba0876b11d8d9c1a34ad352ee77afbfe0da0ddaa45a535dd2"}
Jul 16 22:46:25.995453 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:25.995347    2564 scope.go:117] "RemoveContainer" containerID="7d1d1645de3c730774c10da83550bd65214c3c03826871403fd83f7da88d6822"
Jul 16 22:46:25.995623 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:25.995606    2564 scope.go:117] "RemoveContainer" containerID="165130e9db93462ba0876b11d8d9c1a34ad352ee77afbfe0da0ddaa45a535dd2"
Jul 16 22:46:25.995867 ip-10-0-128-173 kubenswrapper[2564]: E0716 22:46:25.995848    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z_odh-ai-gateway-infra(066575ee-4dd2-4056-b416-31194f7475a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" podUID="066575ee-4dd2-4056-b416-31194f7475a2"
Jul 16 22:46:27.023603 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.023567    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"]
Jul 16 22:46:27.050316 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.050287    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9bdf84c6c-t7md7_0cc69be4-dc26-4fad-a7c4-1b46ecb29369/authorino/0.log"
Jul 16 22:46:27.164345 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.164316    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"
Jul 16 22:46:27.173324 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.173300    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9n45m_a27f3c30-2fe1-4d31-9b35-6aea679a1e00/manager/0.log"
Jul 16 22:46:27.289218 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.289140    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8sznl_49284255-ad2e-40b5-9ab4-916e2c1c69dc/manager/0.log"
Jul 16 22:46:27.312056 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.312030    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066575ee-4dd2-4056-b416-31194f7475a2-maas-api-service-ca\") pod \"066575ee-4dd2-4056-b416-31194f7475a2\" (UID: \"066575ee-4dd2-4056-b416-31194f7475a2\") "
Jul 16 22:46:27.312379 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.312358    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/066575ee-4dd2-4056-b416-31194f7475a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "066575ee-4dd2-4056-b416-31194f7475a2" (UID: "066575ee-4dd2-4056-b416-31194f7475a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:46:27.413040 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.413002    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/066575ee-4dd2-4056-b416-31194f7475a2-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 16 22:46:27.528499 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.528473    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fj82m_e4cc2418-07cf-4b4e-a702-8c4006aeeca1/registry-server/0.log"
Jul 16 22:46:27.659715 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.659640    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-592vz_1d2e9a4d-2f95-4ffd-8490-ce5a04a90580/manager/0.log"
Jul 16 22:46:27.911260 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:27.911183    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kdkrg_f360a550-9cbf-4708-a2ed-aa4338df0be8/manager/0.log"
Jul 16 22:46:28.012216 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.012192    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"
Jul 16 22:46:28.012363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.012234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z" event={"ID":"066575ee-4dd2-4056-b416-31194f7475a2","Type":"ContainerDied","Data":"8f8445ea491df27e46eb691ec81b8e7e3686a8bd8b0f302100734fa332593512"}
Jul 16 22:46:28.012363 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.012295    2564 scope.go:117] "RemoveContainer" containerID="165130e9db93462ba0876b11d8d9c1a34ad352ee77afbfe0da0ddaa45a535dd2"
Jul 16 22:46:28.038264 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.038237    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"]
Jul 16 22:46:28.058460 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.058436    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f757c4eb-ms77z"]
Jul 16 22:46:28.126011 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.125980    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="066575ee-4dd2-4056-b416-31194f7475a2" path="/var/lib/kubelet/pods/066575ee-4dd2-4056-b416-31194f7475a2/volumes"
Jul 16 22:46:28.721930 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.721879    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-695b65bf64-rfgxf_28f7e0b8-1a4d-4187-bc78-058925285ebe/kube-auth-proxy/0.log"
Jul 16 22:46:28.966514 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:28.966478    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d74pp_019c2a8a-65a4-44e7-8b8f-05affe600636/payload-pre-processing/0.log"
Jul 16 22:46:29.086846 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:29.086774    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nxppb_05730b62-84a5-4146-8ff3-4a274b9bd942/payload-processing/0.log"
Jul 16 22:46:29.209136 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:29.209103    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64db4fc64d-wlpxp_d70792ad-fa1c-4f45-be6c-05892fd3fc19/router/0.log"
Jul 16 22:46:29.571099 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:29.571071    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-225gx_0a756350-8d5c-4d92-bfa6-086e21b016a4/main/0.log"
Jul 16 22:46:35.643407 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:35.643365    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:35.770088 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:35.770052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:39.079173 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:39.079146    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zkdf9_5f9adec6-0471-441e-8c76-2dbdd18bfc18/global-pull-secret-syncer/0.log"
Jul 16 22:46:39.167953 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:39.167921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7vc4n_234d30fa-4935-4b4e-9091-9dbc56829631/konnectivity-agent/0.log"
Jul 16 22:46:39.358127 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:39.358050    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-173.ec2.internal_9617a17fc683a92e3fbbd52c642b5794/haproxy/0.log"
Jul 16 22:46:44.053968 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:44.053927    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9bdf84c6c-t7md7_0cc69be4-dc26-4fad-a7c4-1b46ecb29369/authorino/0.log"
Jul 16 22:46:44.098283 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:44.098253    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9n45m_a27f3c30-2fe1-4d31-9b35-6aea679a1e00/manager/0.log"
Jul 16 22:46:44.170725 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:44.170696    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8sznl_49284255-ad2e-40b5-9ab4-916e2c1c69dc/manager/0.log"
Jul 16 22:46:44.261499 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:44.261467    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fj82m_e4cc2418-07cf-4b4e-a702-8c4006aeeca1/registry-server/0.log"
Jul 16 22:46:44.374358 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:44.374277    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-592vz_1d2e9a4d-2f95-4ffd-8490-ce5a04a90580/manager/0.log"
Jul 16 22:46:44.502603 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:44.502566    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kdkrg_f360a550-9cbf-4708-a2ed-aa4338df0be8/manager/0.log"
Jul 16 22:46:46.670127 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:46.670100    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-2f7rl_f6340ec6-1b89-4f62-a863-22358fc973ea/cluster-monitoring-operator/0.log"
Jul 16 22:46:47.271842 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:47.271795    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjrps_3c2df31e-e0d0-4429-a7e9-ec77871a0d9d/node-exporter/0.log"
Jul 16 22:46:47.304943 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:47.304891    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjrps_3c2df31e-e0d0-4429-a7e9-ec77871a0d9d/kube-rbac-proxy/0.log"
Jul 16 22:46:47.334415 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:47.334388    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjrps_3c2df31e-e0d0-4429-a7e9-ec77871a0d9d/init-textfile/0.log"
Jul 16 22:46:48.838928 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.838881    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"]
Jul 16 22:46:48.839312 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.839274    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="066575ee-4dd2-4056-b416-31194f7475a2" containerName="revoke-keys"
Jul 16 22:46:48.839312 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.839286    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="066575ee-4dd2-4056-b416-31194f7475a2" containerName="revoke-keys"
Jul 16 22:46:48.839312 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.839297    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="066575ee-4dd2-4056-b416-31194f7475a2" containerName="revoke-keys"
Jul 16 22:46:48.839312 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.839303    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="066575ee-4dd2-4056-b416-31194f7475a2" containerName="revoke-keys"
Jul 16 22:46:48.839491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.839371    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="066575ee-4dd2-4056-b416-31194f7475a2" containerName="revoke-keys"
Jul 16 22:46:48.839491 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.839383    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="066575ee-4dd2-4056-b416-31194f7475a2" containerName="revoke-keys"
Jul 16 22:46:48.842295 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.842277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:48.845308 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.845282    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zq9hl\"/\"kube-root-ca.crt\""
Jul 16 22:46:48.846269 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.846245    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zq9hl\"/\"openshift-service-ca.crt\""
Jul 16 22:46:48.846377 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.846257    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zq9hl\"/\"default-dockercfg-q7xtq\""
Jul 16 22:46:48.852630 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.852611    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"]
Jul 16 22:46:48.899334 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.899303    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-sys\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:48.899520 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.899338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-lib-modules\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:48.899520 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.899411    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68nqw\" (UniqueName: \"kubernetes.io/projected/99ac9b70-4994-4317-bfbb-16c0fed47b07-kube-api-access-68nqw\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:48.899520 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.899459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-proc\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:48.899520 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:48.899506    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-podres\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000632 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000590    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-sys\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000632 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-lib-modules\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000659    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68nqw\" (UniqueName: \"kubernetes.io/projected/99ac9b70-4994-4317-bfbb-16c0fed47b07-kube-api-access-68nqw\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000685    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-proc\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-podres\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-proc\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000782    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-lib-modules\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.000866 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000731    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-sys\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.001147 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.000893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99ac9b70-4994-4317-bfbb-16c0fed47b07-podres\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.010010 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.009985    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68nqw\" (UniqueName: \"kubernetes.io/projected/99ac9b70-4994-4317-bfbb-16c0fed47b07-kube-api-access-68nqw\") pod \"perf-node-gather-daemonset-7ttq4\" (UID: \"99ac9b70-4994-4317-bfbb-16c0fed47b07\") " pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.153976 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.153872    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:49.293280 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.293256    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"]
Jul 16 22:46:49.295615 ip-10-0-128-173 kubenswrapper[2564]: W0716 22:46:49.295592    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod99ac9b70_4994_4317_bfbb_16c0fed47b07.slice/crio-0b4bed0442da3c1d9772248c6246433b9943592fbe0499d8031e43c16c266421 WatchSource:0}: Error finding container 0b4bed0442da3c1d9772248c6246433b9943592fbe0499d8031e43c16c266421: Status 404 returned error can't find the container with id 0b4bed0442da3c1d9772248c6246433b9943592fbe0499d8031e43c16c266421
Jul 16 22:46:49.315950 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:49.315928    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-nj2nn_88805df0-8f06-432d-9316-c97536c8bc08/networking-console-plugin/0.log"
Jul 16 22:46:50.005773 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:50.005730    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/1.log"
Jul 16 22:46:50.010160 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:50.010134    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-77j96_cb9b083e-7066-40e4-bce1-366de4032eaa/console-operator/2.log"
Jul 16 22:46:50.133572 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:50.133536    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4" event={"ID":"99ac9b70-4994-4317-bfbb-16c0fed47b07","Type":"ContainerStarted","Data":"5e8e80766293fc3ed9b5902326544989f04aa5eb9d2a0c3d487051a8c213d701"}
Jul 16 22:46:50.133758 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:50.133584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4" event={"ID":"99ac9b70-4994-4317-bfbb-16c0fed47b07","Type":"ContainerStarted","Data":"0b4bed0442da3c1d9772248c6246433b9943592fbe0499d8031e43c16c266421"}
Jul 16 22:46:50.133758 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:50.133668    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:50.152981 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:50.152890    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4" podStartSLOduration=2.15287593 podStartE2EDuration="2.15287593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:46:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:46:50.150574881 +0000 UTC m=+3476.539848667" watchObservedRunningTime="2026-07-16 22:46:50.15287593 +0000 UTC m=+3476.542149742"
Jul 16 22:46:51.376932 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:51.376883    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-r4frb_98296b74-f3c7-4a28-aba5-fd0bfab5d32f/volume-data-source-validator/0.log"
Jul 16 22:46:52.763429 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:52.763395    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wn4d2_2e635ee7-8ae6-4a90-a66f-e33cfd434838/dns/0.log"
Jul 16 22:46:52.794138 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:52.794106    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wn4d2_2e635ee7-8ae6-4a90-a66f-e33cfd434838/kube-rbac-proxy/0.log"
Jul 16 22:46:52.881588 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:52.881554    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s8d4d_6cdca38f-6c5c-4ab2-a93d-779ac9a7aa41/dns-node-resolver/0.log"
Jul 16 22:46:53.486294 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:53.486256    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nwfwh_e9c0dc4b-aa01-4b81-a48d-4cd6584e3c4e/node-ca/0.log"
Jul 16 22:46:54.722971 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:54.722942    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-695b65bf64-rfgxf_28f7e0b8-1a4d-4187-bc78-058925285ebe/kube-auth-proxy/0.log"
Jul 16 22:46:54.826776 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:54.826741    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d74pp_019c2a8a-65a4-44e7-8b8f-05affe600636/payload-pre-processing/0.log"
Jul 16 22:46:54.859582 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:54.859551    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nxppb_05730b62-84a5-4146-8ff3-4a274b9bd942/payload-processing/0.log"
Jul 16 22:46:54.881528 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:54.881502    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64db4fc64d-wlpxp_d70792ad-fa1c-4f45-be6c-05892fd3fc19/router/0.log"
Jul 16 22:46:55.594100 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:55.594067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8pnfs_3b323658-e5c3-4845-9038-a8695bfed36f/serve-healthcheck-canary/0.log"
Jul 16 22:46:56.138278 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:56.138242    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-trdvm_1173fb03-41c1-4aad-902f-7b7f6d9bd371/insights-operator/0.log"
Jul 16 22:46:56.138957 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:56.138938    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-trdvm_1173fb03-41c1-4aad-902f-7b7f6d9bd371/insights-operator/1.log"
Jul 16 22:46:56.149261 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:56.149228    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zq9hl/perf-node-gather-daemonset-7ttq4"
Jul 16 22:46:56.164121 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:56.164088    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9dtrc_38f4b3d2-9161-45a6-ad8c-cc0c509d65a4/kube-rbac-proxy/0.log"
Jul 16 22:46:56.196568 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:56.196537    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9dtrc_38f4b3d2-9161-45a6-ad8c-cc0c509d65a4/exporter/0.log"
Jul 16 22:46:56.225528 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:56.225487    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9dtrc_38f4b3d2-9161-45a6-ad8c-cc0c509d65a4/extractor/0.log"
Jul 16 22:46:58.813635 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:58.813582    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tlhk5_cd872a79-78f1-4c79-b884-6b2723da3867/manager/0.log"
Jul 16 22:46:58.954611 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:58.954581    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bvzc8_0653006a-640b-4264-be65-9c922d3d7f3a/manager/0.log"
Jul 16 22:46:59.067659 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:59.067625    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f4d5cdddc-wqmfp_241531e3-1b0f-492e-a104-d49454633c50/manager/0.log"
Jul 16 22:46:59.209591 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:59.209515    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k_58d9f09a-6837-4e79-b0c0-78f830eeb47d/manager/0.log"
Jul 16 22:46:59.231981 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:46:59.231952    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54cf7dfc9c-p2t5k_58d9f09a-6837-4e79-b0c0-78f830eeb47d/copy-manifests/0.log"
Jul 16 22:47:00.260582 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:00.260552    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-695b65bf64-rfgxf_28f7e0b8-1a4d-4187-bc78-058925285ebe/kube-auth-proxy/0.log"
Jul 16 22:47:00.353593 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:00.353563    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d74pp_019c2a8a-65a4-44e7-8b8f-05affe600636/payload-pre-processing/0.log"
Jul 16 22:47:00.377738 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:00.377707    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nxppb_05730b62-84a5-4146-8ff3-4a274b9bd942/payload-processing/0.log"
Jul 16 22:47:00.398154 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:00.398110    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64db4fc64d-wlpxp_d70792ad-fa1c-4f45-be6c-05892fd3fc19/router/0.log"
Jul 16 22:47:02.004979 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:02.004951    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-695b65bf64-rfgxf_28f7e0b8-1a4d-4187-bc78-058925285ebe/kube-auth-proxy/0.log"
Jul 16 22:47:02.131393 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:02.131356    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-d74pp_019c2a8a-65a4-44e7-8b8f-05affe600636/payload-pre-processing/0.log"
Jul 16 22:47:02.164052 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:02.164017    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-nxppb_05730b62-84a5-4146-8ff3-4a274b9bd942/payload-processing/0.log"
Jul 16 22:47:02.191094 ip-10-0-128-173 kubenswrapper[2564]: I0716 22:47:02.191065    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64db4fc64d-wlpxp_d70792ad-fa1c-4f45-be6c-05892fd3fc19/router/0.log"