Jul 10 13:26:14.303556 ip-10-0-135-59 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 13:26:14.678590 ip-10-0-135-59 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 13:26:14.678590 ip-10-0-135-59 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 13:26:14.678590 ip-10-0-135-59 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 13:26:14.678590 ip-10-0-135-59 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 13:26:14.678590 ip-10-0-135-59 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 13:26:14.679487 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.679413    2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 13:26:14.684642 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684628    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 13:26:14.684642 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684642    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684646    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684649    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684653    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684657    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684662    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684665    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684668    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684671    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684674    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684677    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684679    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684682    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684684    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684687    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684689    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684692    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684694    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684697    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 13:26:14.684699 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684699    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684702    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684705    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684707    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684710    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684713    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684715    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684718    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684720    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684723    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684727    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684730    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684733    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684736    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684739    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684743    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684746    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684748    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684751    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684754    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 13:26:14.685161 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684756    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684759    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684761    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684763    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684766    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684768    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684770    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684773    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684775    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684778    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684781    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684784    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684786    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684789    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684792    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684795    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684797    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684800    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684802    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684805    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 13:26:14.685640 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684807    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684809    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684812    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684815    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684817    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684819    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684822    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684825    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684829    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684833    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684836    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684839    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684842    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684844    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684846    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684849    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684851    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684854    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684856    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 13:26:14.686155 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684859    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684861    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684864    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684866    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684869    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684872    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684874    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684877    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684879    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684882    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684885    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684887    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684890    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684892    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684895    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684898    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684901    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684903    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684905    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.684908    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 13:26:14.686607 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685623    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685631    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685633    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685636    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685638    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685641    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685644    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685646    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685650    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685652    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685655    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685659    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685663    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685666    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685669    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685671    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685674    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685677    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685680    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685683    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 13:26:14.687105 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685687    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685690    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685693    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685696    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685699    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685702    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685705    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685708    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685711    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685713    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685716    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685719    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685721    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685723    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685727    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685729    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685732    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685734    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685737    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685739    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 13:26:14.687606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685742    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685744    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685747    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685750    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685752    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685755    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685758    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685760    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685763    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685765    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685768    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685770    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685773    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685776    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685778    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685780    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685783    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685785    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685788    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685790    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 13:26:14.688123 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685792    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685795    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685797    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685801    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685805    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685808    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685811    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685814    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685817    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685820    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685822    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685825    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685828    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685830    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685833    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685835    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685838    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685840    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 13:26:14.688619 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685843    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685845    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685848    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685851    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685854    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685857    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685859    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685862    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685864    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685866    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685869    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685871    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685873    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685876    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685878    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685881    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685884    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685886    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685889    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685891    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 13:26:14.689028 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.685893    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686229    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686233    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686236    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686239    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686241    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686244    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686247    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686250    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686252    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686255    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686257    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686260    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686262    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686265    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686268    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686271    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686273    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686276    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686278    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 13:26:14.689511 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686281    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686283    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686286    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686288    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686290    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686293    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686295    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686298    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686316    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686319    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686322    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686324    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686327    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686329    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686332    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686335    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686338    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686340    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686343    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686346    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 13:26:14.689975 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686348    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686351    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686353    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686356    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686359    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686362    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686364    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686370    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686372    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686375    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686378    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686380    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686383    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686386    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686389    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686392    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686394    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686397    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686400    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686402    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 13:26:14.690451 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686405    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686407    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686410    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686412    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686415    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686418    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686420    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686423    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686426    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686428    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686431    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686435    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686439    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686442    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686446    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686450    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686454    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686457    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 13:26:14.690928 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686460    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686464    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686467    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686469    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686472    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686474    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686484    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686487    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686491    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686493    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686496    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686498    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686501    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686503    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686506    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686508    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686511    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686513    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686516    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686519    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 13:26:14.691368 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686523    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.686525    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686593    2554 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686599    2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686606    2554 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686610    2554 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686615    2554 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686618    2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686622    2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686626    2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686629    2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686632    2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686635    2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686640    2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686643    2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686646    2554 flags.go:64] FLAG: --cgroup-root=""
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686649    2554 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686652    2554 flags.go:64] FLAG: --client-ca-file=""
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686655    2554 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686658    2554 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686662    2554 flags.go:64] FLAG: --cluster-domain=""
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686664    2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686667    2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686671    2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 13:26:14.691834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686674    2554 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686678    2554 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686680    2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686683    2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686687    2554 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686690    2554 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686693    2554 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686696    2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686700    2554 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686703    2554 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686707    2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686711    2554 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686713    2554 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686716    2554 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686719    2554 flags.go:64] FLAG: --enable-server="true"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686722    2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686727    2554 flags.go:64] FLAG: --event-burst="100"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686730    2554 flags.go:64] FLAG: --event-qps="50"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686732    2554 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686735    2554 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686738    2554 flags.go:64] FLAG: --eviction-hard=""
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686743    2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686746    2554 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686749    2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686752    2554 flags.go:64] FLAG: --eviction-soft=""
Jul 10 13:26:14.692408 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686755    2554 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686757    2554 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686760    2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686763    2554 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686766    2554 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686769    2554 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686771    2554 flags.go:64] FLAG: --feature-gates=""
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686775    2554 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686777    2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686780    2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686783    2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686786    2554 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686789    2554 flags.go:64] FLAG: --help="false"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686792    2554 flags.go:64] FLAG: --hostname-override="ip-10-0-135-59.ec2.internal"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686795    2554 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686799    2554 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686802    2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686806    2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686809    2554 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686812    2554 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686815    2554 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686818    2554 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686820    2554 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 13:26:14.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686823    2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686826    2554 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686829    2554 flags.go:64] FLAG: --kube-reserved=""
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686832    2554 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686834    2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686839    2554 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686842    2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686845    2554 flags.go:64] FLAG: --lock-file=""
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686847    2554 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686850    2554 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686853    2554 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686858    2554 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686861    2554 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686864    2554 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686867    2554 flags.go:64] FLAG: --logging-format="text"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686869    2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686872    2554 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686875    2554 flags.go:64] FLAG: --manifest-url=""
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686878    2554 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686882    2554 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686885    2554 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686889    2554 flags.go:64] FLAG: --max-pods="110"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686901    2554 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686905    2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686909    2554 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 13:26:14.693800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686912    2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686915    2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686918    2554 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686921    2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686929    2554 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686932    2554 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686935    2554 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686938    2554 flags.go:64] FLAG: --pod-cidr=""
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686940    2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686946    2554 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686948    2554 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686951    2554 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686956    2554 flags.go:64] FLAG: --port="10250"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686959    2554 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686962    2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f072503c9242e38"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686965    2554 flags.go:64] FLAG: --qos-reserved=""
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686967    2554 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686970    2554 flags.go:64] FLAG: --register-node="true"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686973    2554 flags.go:64] FLAG: --register-with-taints=""
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686976    2554 flags.go:64] FLAG: --registry-burst="10"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686979    2554 flags.go:64] FLAG: --registry-qps="5"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686982    2554 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686984    2554 flags.go:64] FLAG: --reserved-memory=""
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686988    2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686991    2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 13:26:14.694831 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686995    2554 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.686997    2554 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687000    2554 flags.go:64] FLAG: --runonce="false"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687003    2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687006    2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687009    2554 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687012    2554 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687017    2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687020    2554 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687024    2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687027    2554 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687030    2554 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687033    2554 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687036    2554 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687038    2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687041    2554 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687044    2554 flags.go:64] FLAG: --system-cgroups=""
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687047    2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687052    2554 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687056    2554 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687059    2554 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687063    2554 flags.go:64] FLAG: --tls-min-version=""
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687066    2554 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687068    2554 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687071    2554 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 13:26:14.695682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687074    2554 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687077    2554 flags.go:64] FLAG: --v="2"
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687080    2554 flags.go:64] FLAG: --version="false"
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687084    2554 flags.go:64] FLAG: --vmodule=""
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687087    2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.687091    2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687257    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687273    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687277    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687280    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687284    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687286    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687289    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687292    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687295    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687298    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687300    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687317    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687322    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687324    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687327    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 13:26:14.696348 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687330    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687332    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687335    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687338    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687340    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687343    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687346    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687348    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687351    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687354    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687356    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687361    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687366    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687370    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687374    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687377    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687380    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687383    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687387    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 13:26:14.696837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687390    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687393    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687396    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687399    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687401    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687404    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687407    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687409    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687412    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687415    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687417    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687420    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687422    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687425    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687427    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687430    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687433    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687436    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687438    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687441    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 13:26:14.697283 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687443    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687446    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687449    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687452    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687454    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687457    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687459    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687462    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687465    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687467    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687470    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687473    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687476    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687479    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687482    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687484    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687487    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687490    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687492    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 13:26:14.697819 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687495    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687497    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687500    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687502    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687505    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687507    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687510    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687513    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687515    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687517    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687520    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687523    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687525    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687528    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687531    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687533    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687535    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687538    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687541    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687543    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 13:26:14.698275 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687546    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687548    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687551    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687554    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687557    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.687560    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.688142    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.696958    2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.696975    2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.697833    2554 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.697841    2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697901    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697907    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697911    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697914    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 13:26:14.698751 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697917    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697920    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697923    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697926    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697929    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697931    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697934    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697937    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697940    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697944    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697948    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697951    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697954    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697956    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697959    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697962    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697965    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697968    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697970    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 13:26:14.699122 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697974    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697976    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697979    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697983    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697986    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697989    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697992    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697995    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.697997    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698000    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698003    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698005    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698008    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698010    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698012    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698015    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698018    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698020    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698022    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698025    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698027    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 13:26:14.699598 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698030    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698032    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698036    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698038    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698041    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698043    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698046    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698049    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698052    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698055    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698057    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698060    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698062    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698065    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698067    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698070    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698073    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698076    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698079    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 13:26:14.700081 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698082    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698084    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698087    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698090    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698092    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698095    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698097    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698100    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698102    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698105    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698107    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698110    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698112    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698116    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698119    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698122    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698125    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698128    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698130    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 13:26:14.700677 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698133    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698136    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698138    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698142    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698145    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698148    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698151    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698153    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698156    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698158    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698161    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698164    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698167    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698169    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698172    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698174    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 13:26:14.701131 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698177    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.698182    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698325    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698331    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698334    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698336    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698340    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698344    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698349    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698352    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698355    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698358    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698360    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698363    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698366    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 13:26:14.701539 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698369    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698372    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698375    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698377    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698380    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698382    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698385    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698387    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698389    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698392    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698395    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698397    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698400    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698402    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698405    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698407    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698410    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698412    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698415    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698417    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 13:26:14.701901 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698420    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698422    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698425    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698427    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698430    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698432    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698434    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698437    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698439    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698442    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698444    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698446    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698450    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698453    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698455    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698457    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698460    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698462    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698465    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698467    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 13:26:14.702410 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698469    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698472    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698474    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698477    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698479    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698482    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698484    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698487    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698489    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698492    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698494    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698497    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698499    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698501    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698504    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698506    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698509    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698511    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698513    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 13:26:14.702867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698516    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698519    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698522    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698525    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698527    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698530    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698533    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698536    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698538    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698540    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698543    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698545    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698548    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698550    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698553    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698555    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698557    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698560    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698563    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698565    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 13:26:14.703436 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698568    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698570    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698573    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698575    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698578    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698580    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:14.698583    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.698587    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.698691    2554 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.701784    2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.703356    2554 server.go:1019] "Starting client certificate rotation"
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.703448    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 13:26:14.703940 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.703482    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 13:26:14.706025 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.706011    2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 13:26:14.708658 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.708641    2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 13:26:14.720434 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:14.720412    2554 log.go:25] "Validated CRI v1 runtime API"
Jul 10 13:26:16.441632 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.441614    2554 log.go:25] "Validated CRI v1 image API"
Jul 10 13:26:16.442865 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.442851    2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 13:26:16.445962 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.445945    2554 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8b303159-6b8a-4df9-999e-eb0cdc64a1ef:/dev/nvme0n1p4 db298ffe-1646-452a-bb57-f92e161c6a16:/dev/nvme0n1p3]
Jul 10 13:26:16.446026 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.445962    2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 13:26:16.452092 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.451795    2554 manager.go:216] Machine: {Timestamp:2026-07-10 13:26:16.449558159 +0000 UTC m=+0.315684934 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097837 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e5f9d2693666cd636c10ef3928e7e SystemUUID:ec2e5f9d-2693-666c-d636-c10ef3928e7e BootID:44bcb781-0693-4192-a2c7-92282ae1dc2b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:62:98:b2:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:62:98:b2:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:8e:13:e4:12:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 13:26:16.452092 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.452083    2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 13:26:16.452214 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.452202    2554 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 13:26:16.453295 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.453272    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 13:26:16.455327 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.455302    2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 13:26:16.455469 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.455330    2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-59.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 13:26:16.456524 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.456513    2554 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 13:26:16.456562 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.456528    2554 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 13:26:16.456562 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.456536    2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 13:26:16.456562 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.456549    2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 13:26:16.458312 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.458302    2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 13:26:16.461652 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.461641    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 13:26:16.461893 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.461882    2554 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 13:26:16.461925 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.461913    2554 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 13:26:16.461925 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.461922    2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 13:26:16.461983 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.461945    2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 13:26:16.461983 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.461954    2554 kubelet.go:398] "Adding apiserver pod source"
Jul 10 13:26:16.461983 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.461962    2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 13:26:16.462311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.462300    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 13:26:16.462351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.462319    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 13:26:16.464788 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.464770    2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 13:26:16.467001 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.466989    2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 13:26:16.467051 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467018    2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 13:26:16.467170 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467159    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 13:26:16.467203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467178    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 13:26:16.467203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467184    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 13:26:16.467203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467189    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 13:26:16.467203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467196    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 13:26:16.467203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467201    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 13:26:16.467596 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467588    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 13:26:16.467596 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467596    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 13:26:16.467655 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467603    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 13:26:16.467655 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467608    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 13:26:16.467655 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467617    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 13:26:16.467655 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.467625    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 13:26:16.469295 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.469283    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 13:26:16.469334 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.469299    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 13:26:16.472272 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.472240    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 13:26:16.472272 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.472264    2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-59.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 13:26:16.472426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.472277    2554 server.go:1268] "Started kubelet"
Jul 10 13:26:16.472426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.472376    2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 13:26:16.472426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.472383    2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 13:26:16.472541 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.472450    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 13:26:16.472541 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.472457    2554 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 13:26:16.473179 ip-10-0-135-59 systemd[1]: Started Kubernetes Kubelet.
Jul 10 13:26:16.473976 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.473809    2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 13:26:16.474360 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.474349    2554 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 13:26:16.477791 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.476915    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-59.ec2.internal.18c0f0954d0db60d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-59.ec2.internal,UID:ip-10-0-135-59.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-59.ec2.internal,},FirstTimestamp:2026-07-10 13:26:16.472253965 +0000 UTC m=+0.338380609,LastTimestamp:2026-07-10 13:26:16.472253965 +0000 UTC m=+0.338380609,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-59.ec2.internal,}"
Jul 10 13:26:16.478161 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.478142    2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 13:26:16.478670 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.478656    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gm7zj"
Jul 10 13:26:16.481478 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.481463    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 13:26:16.481546 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.481469    2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 13:26:16.481593 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.481558    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:16.482186 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482171    2554 factory.go:55] Registering systemd factory
Jul 10 13:26:16.482280 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482210    2554 factory.go:223] Registration of the systemd container factory successfully
Jul 10 13:26:16.482280 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482237    2554 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 13:26:16.482280 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482252    2554 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 13:26:16.482437 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482323    2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 13:26:16.482437 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482326    2554 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 13:26:16.482437 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482333    2554 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 13:26:16.482437 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482405    2554 factory.go:153] Registering CRI-O factory
Jul 10 13:26:16.482437 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482416    2554 factory.go:223] Registration of the crio container factory successfully
Jul 10 13:26:16.482579 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482490    2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 13:26:16.482579 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482513    2554 factory.go:103] Registering Raw factory
Jul 10 13:26:16.482579 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482530    2554 manager.go:1196] Started watching for new ooms in manager
Jul 10 13:26:16.482954 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.482940    2554 manager.go:349] Starting recovery of all containers
Jul 10 13:26:16.484026 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.483997    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 13:26:16.484125 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.484089    2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-59.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 13:26:16.485595 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.485569    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gm7zj"
Jul 10 13:26:16.493836 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.493797    2554 manager.go:354] Recovery completed
Jul 10 13:26:16.498618 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.498606    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 13:26:16.500367 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.500352    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 13:26:16.500442 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.500378    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 13:26:16.500442 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.500387    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientPID"
Jul 10 13:26:16.500938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.500913    2554 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 13:26:16.500938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.500927    2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 13:26:16.500938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.500938    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 13:26:16.501089 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.500942    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 13:26:16.501541 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.501527    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 13:26:16.501580 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.501553    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 13:26:16.501580 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.501566    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientPID"
Jul 10 13:26:16.502050 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.502039    2554 policy_none.go:49] "None policy: Start"
Jul 10 13:26:16.502083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.502053    2554 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 13:26:16.502083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.502063    2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 13:26:16.503159 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.503149    2554 policy_none.go:47] "Start"
Jul 10 13:26:16.545421 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.545404    2554 manager.go:337] "Starting Device Plugin manager"
Jul 10 13:26:16.545496 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.545485    2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 13:26:16.545556 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.545499    2554 server.go:87] "Starting device plugin registration server"
Jul 10 13:26:16.545771 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.545757    2554 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 13:26:16.545857 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.545772    2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 13:26:16.545931 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.545877    2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 13:26:16.545981 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.545967    2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 13:26:16.545981 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.545981    2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 13:26:16.546403 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.546386    2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 13:26:16.546476 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.546434    2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:16.646305 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.646248    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 13:26:16.647357 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.647267    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 13:26:16.647357 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.647296    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 13:26:16.647357 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.647306    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientPID"
Jul 10 13:26:16.647357 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.647337    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.653929 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.653915    2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.653978 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.653934    2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-59.ec2.internal\": node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:16.656277 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.656258    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 13:26:16.657532 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.657515    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 13:26:16.657616 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.657554    2554 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 13:26:16.657688 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.657616    2554 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 13:26:16.657688 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.657652    2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 13:26:16.659224 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.659207    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 13:26:16.667676 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.667654    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:16.758708 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.758677    2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"]
Jul 10 13:26:16.758784 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.758750    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 13:26:16.759574 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.759561    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 13:26:16.759638 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.759586    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 13:26:16.759638 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.759599    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientPID"
Jul 10 13:26:16.760929 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.760917    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 13:26:16.761005 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.760992    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.761051 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.761017    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 13:26:16.761508 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.761495    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 13:26:16.761572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.761522    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 13:26:16.761572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.761531    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 13:26:16.761572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.761540    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientPID"
Jul 10 13:26:16.761572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.761550    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 13:26:16.761572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.761561    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientPID"
Jul 10 13:26:16.762730 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.762712    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.762811 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.762742    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 13:26:16.763353 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.763338    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 13:26:16.763438 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.763362    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 13:26:16.763438 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.763372    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeHasSufficientPID"
Jul 10 13:26:16.767777 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.767759    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:16.778403 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.778381    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-59.ec2.internal\" not found" node="ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.781637 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.781623    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-59.ec2.internal\" not found" node="ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.783752 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.783733    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9020b496f2ad10a4002b5e25dcae18a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal\" (UID: \"9020b496f2ad10a4002b5e25dcae18a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.783799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.783759    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9020b496f2ad10a4002b5e25dcae18a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal\" (UID: \"9020b496f2ad10a4002b5e25dcae18a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.783799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.783778    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcfe08aec5c2bd135c282f4562048962-config\") pod \"kube-apiserver-proxy-ip-10-0-135-59.ec2.internal\" (UID: \"fcfe08aec5c2bd135c282f4562048962\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.867850 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.867812    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:16.884687 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.884666    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9020b496f2ad10a4002b5e25dcae18a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal\" (UID: \"9020b496f2ad10a4002b5e25dcae18a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.884761 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.884696    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9020b496f2ad10a4002b5e25dcae18a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal\" (UID: \"9020b496f2ad10a4002b5e25dcae18a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.884761 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.884717    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcfe08aec5c2bd135c282f4562048962-config\") pod \"kube-apiserver-proxy-ip-10-0-135-59.ec2.internal\" (UID: \"fcfe08aec5c2bd135c282f4562048962\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.884761 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.884746    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcfe08aec5c2bd135c282f4562048962-config\") pod \"kube-apiserver-proxy-ip-10-0-135-59.ec2.internal\" (UID: \"fcfe08aec5c2bd135c282f4562048962\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.884888 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.884772    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9020b496f2ad10a4002b5e25dcae18a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal\" (UID: \"9020b496f2ad10a4002b5e25dcae18a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.884888 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:16.884778    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9020b496f2ad10a4002b5e25dcae18a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal\" (UID: \"9020b496f2ad10a4002b5e25dcae18a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:16.968728 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:16.968701    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.069228 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.069213    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.081394 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.081372    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:17.086064 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.085768    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:17.169442 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.169418    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.269906 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.269863    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.370384 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.370363    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.411549 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.411528    2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 13:26:17.418911 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.418893    2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 13:26:17.419019 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.418999    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection force closed via ClientConn.Close\") has prevented the request from succeeding"
Jul 10 13:26:17.419069 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.419004    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection force closed via ClientConn.Close\") has prevented the request from succeeding"
Jul 10 13:26:17.471437 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.471415    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.490406 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.490362    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 13:21:16 +0000 UTC" deadline="2028-03-15 11:57:37.835330407 +0000 UTC"
Jul 10 13:26:17.490406 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.490404    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14734h31m20.344930402s"
Jul 10 13:26:17.572404 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.572353    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.672916 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.672887    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.687945 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.687645    2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 13:26:17.689188 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:17.689161    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9020b496f2ad10a4002b5e25dcae18a0.slice/crio-de6e35d29699326101f572873fc4b2b58950069c08d1b3545cdd9c3d1c185ffe WatchSource:0}: Error finding container de6e35d29699326101f572873fc4b2b58950069c08d1b3545cdd9c3d1c185ffe: Status 404 returned error can't find the container with id de6e35d29699326101f572873fc4b2b58950069c08d1b3545cdd9c3d1c185ffe
Jul 10 13:26:17.689627 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:17.689604    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfe08aec5c2bd135c282f4562048962.slice/crio-05b0a261db6c7736d16f29559d58c6a379a6e2aa1810f11426c99432497fd047 WatchSource:0}: Error finding container 05b0a261db6c7736d16f29559d58c6a379a6e2aa1810f11426c99432497fd047: Status 404 returned error can't find the container with id 05b0a261db6c7736d16f29559d58c6a379a6e2aa1810f11426c99432497fd047
Jul 10 13:26:17.694487 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:17.694472    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 13:26:17.773794 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.773771    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.874318 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.874263    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:17.974970 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:17.974942    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:18.075500 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.075466    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:18.175976 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.175942    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:18.276587 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.276557    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:18.377222 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.377193    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-59.ec2.internal\" not found"
Jul 10 13:26:18.422369 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.422346    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 13:26:18.462572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.462512    2554 apiserver.go:52] "Watching apiserver"
Jul 10 13:26:18.472407 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.472386    2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 13:26:18.472834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.472791    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4","openshift-cluster-node-tuning-operator/tuned-gjvql","openshift-image-registry/node-ca-kv96s","openshift-multus/multus-additional-cni-plugins-cmp9c","openshift-ovn-kubernetes/ovnkube-node-pwjcx","kube-system/konnectivity-agent-l69wk","openshift-dns/node-resolver-g4sfq","openshift-multus/multus-zjqnn","openshift-multus/network-metrics-daemon-4qt4r","openshift-network-diagnostics/network-check-target-8swsb","openshift-network-operator/iptables-alerter-j5x7m","kube-system/global-pull-secret-syncer-ggn9l"]
Jul 10 13:26:18.474361 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.474340    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.476087 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.476061    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.477152 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.477132    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 13:26:18.477263 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.477194    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pckl6\""
Jul 10 13:26:18.477263 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.477245    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 13:26:18.477373 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.477198    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 13:26:18.477423 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.477408    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.478243 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.478205    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 13:26:18.478448 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.478431    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 13:26:18.478602 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.478584    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-thvzd\""
Jul 10 13:26:18.479792 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.479772    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.480983 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.480964    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 13:26:18.481137 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.481090    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 13:26:18.481137 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.481122    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 13:26:18.481246 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.481094    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mpn6r\""
Jul 10 13:26:18.481925 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.481901    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 13:26:18.482064 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.482046    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.482135 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.481903    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:18.482378 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.482054    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 13:26:18.482808 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.482788    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 13:26:18.483247 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.483228    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 13:26:18.483335 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.483268    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tnxgd\""
Jul 10 13:26:18.483501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.483481    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.483996 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.483965    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 13:26:18.484081 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.484040    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 13:26:18.484577 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.484421    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 13:26:18.485098 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.484869    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 13:26:18.485098 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.485031    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 13:26:18.485098 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.485073    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 13:26:18.485370 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.485354    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.486446 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.485914    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 13:26:18.486446 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.486107    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 13:26:18.486446 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.486248    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 13:26:18.486446 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.486292    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qt66d\""
Jul 10 13:26:18.486446 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.486341    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 13:26:18.486754 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.486561    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t4ktx\""
Jul 10 13:26:18.487037 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.487021    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.487677 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.487656    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kp9pb\""
Jul 10 13:26:18.487932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.487911    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 13:26:18.489049 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.489029    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 13:26:18.489577 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.489560    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:18.489664 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.489615    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 13:26:18.489969 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.489665    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:18.489969 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.489808    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cpdf7\""
Jul 10 13:26:18.492561 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.492542    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:18.492647 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.492620    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:18.494430 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494412    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.494637 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494611    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysconfig\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.494720 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494647    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.494720 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494673    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.494720 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494696    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22c6k\" (UniqueName: \"kubernetes.io/projected/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-kube-api-access-22c6k\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.494880 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494718    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-socket-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.494934 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494910    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njlhf\" (UniqueName: \"kubernetes.io/projected/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-kube-api-access-njlhf\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.494990 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494938    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-modprobe-d\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.494990 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494963    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-etc-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.494990 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.494985    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-ovn\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495136 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495009    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-kubelet\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495136 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495044    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495136 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495074    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-slash\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495136 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495095    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e-agent-certs\") pod \"konnectivity-agent-l69wk\" (UID: \"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e\") " pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.495136 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495119    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.495354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495140    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.495354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495175    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-os-release\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.495354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495199    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovnkube-script-lib\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495222    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-log-socket\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495243    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-registration-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.495354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495266    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-run\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.495354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495321    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495356    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-tuned\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495378    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-serviceca\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495400    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-cni-netd\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495438    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-env-overrides\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495489    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e-konnectivity-ca\") pod \"konnectivity-agent-l69wk\" (UID: \"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e\") " pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495513    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysctl-conf\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495536    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-var-lib-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495557    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-node-log\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495579    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495602    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-systemd\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495623    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-system-cni-dir\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495645    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-sys\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495672    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-var-lib-kubelet\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.495699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495695    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a88267a2-ec84-4eb3-a673-c6ca74aaf192-tmp\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495718    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495743    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcq46\" (UniqueName: \"kubernetes.io/projected/a88267a2-ec84-4eb3-a673-c6ca74aaf192-kube-api-access-kcq46\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495766    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495790    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-systemd-units\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495811    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-cni-bin\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495851    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-device-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495874    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cnibin\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495902    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovn-node-metrics-cert\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495924    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysctl-d\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495944    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-host\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495967    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-host\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.495984    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496000    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh6tc\" (UniqueName: \"kubernetes.io/projected/186e59bb-668c-4cb7-b099-f0f86ee14eaf-kube-api-access-vh6tc\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496037    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-run-netns\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.496039    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:18.496336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496114    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovnkube-config\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.497060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496148    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-sys-fs\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.497060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496270    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-kubernetes\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.497060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496296    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-lib-modules\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.497060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496319    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-systemd\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.497060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.496353    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2tqr\" (UniqueName: \"kubernetes.io/projected/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-kube-api-access-p2tqr\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.497212 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.497124    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 13:26:18.497212 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.497137    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 13:26:18.497326 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.497293    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q9nbk\""
Jul 10 13:26:18.497407 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.497341    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 13:26:18.498689 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.498622    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal"]
Jul 10 13:26:18.498981 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.498961    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 13:26:18.499074 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.499030    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"
Jul 10 13:26:18.502407 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.502387    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 13:26:18.509326 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.509305    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 13:26:18.509506 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.509490    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal"]
Jul 10 13:26:18.523491 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.523473    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9jtq"
Jul 10 13:26:18.534214 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.534181    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9jtq"
Jul 10 13:26:18.583107 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.583091    2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 13:26:18.596714 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596689    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-var-lib-kubelet\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.596811 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596726    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a88267a2-ec84-4eb3-a673-c6ca74aaf192-tmp\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.596811 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596744    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.596938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596812    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-var-lib-kubelet\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.596938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596855    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-socket-dir-parent\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.596938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596884    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60c13910-366d-437c-b0a1-e3c83a770325-host-slash\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.596938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596913    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcq46\" (UniqueName: \"kubernetes.io/projected/a88267a2-ec84-4eb3-a673-c6ca74aaf192-kube-api-access-kcq46\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596942    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.597112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596967    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-systemd-units\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.596990    2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 13:26:18.597112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597015    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-systemd-units\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597012    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-cni-bin\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597083    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-cni-bin\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597131    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-cni-multus\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597162    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-device-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597189    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cnibin\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597213    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovn-node-metrics-cert\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597239    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-conf-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597254    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-device-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597263    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysctl-d\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597289    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-host\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597286    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cnibin\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597347    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-host\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597396 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597373    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597413    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysctl-d\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597425    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-host\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597456    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh6tc\" (UniqueName: \"kubernetes.io/projected/186e59bb-668c-4cb7-b099-f0f86ee14eaf-kube-api-access-vh6tc\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597466    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597482    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-run-netns\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597506    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovnkube-config\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597519    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-host\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597535    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-run-netns\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597534    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/60c13910-366d-437c-b0a1-e3c83a770325-iptables-alerter-script\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597577    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-sys-fs\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597601    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-kubernetes\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597624    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-lib-modules\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597647    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-systemd\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597689    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-sys-fs\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597719    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-kubernetes\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597723    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-systemd\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.597909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.597778    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-lib-modules\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598058    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovnkube-config\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598213    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2tqr\" (UniqueName: \"kubernetes.io/projected/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-kube-api-access-p2tqr\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598262    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc717d08-6567-4c6b-9cdd-1d260730c235-hosts-file\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598285    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc717d08-6567-4c6b-9cdd-1d260730c235-tmp-dir\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598309    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-cnibin\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598330    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-os-release\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598354    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/047ade14-fb4a-4623-a579-087c0ed54327-cni-binary-copy\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598375    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-hostroot\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598399    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0acdcc26-44da-455e-a2df-a871c3c9a79f-kubelet-config\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598424    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysconfig\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598451    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598475    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598489    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysconfig\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598506    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrcr\" (UniqueName: \"kubernetes.io/projected/ce6950c4-e47b-45eb-9ad1-82ae182964b1-kube-api-access-8hrcr\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598543    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-netns\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598570    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-cni-bin\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.598766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598595    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-multus-certs\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598625    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs28q\" (UniqueName: \"kubernetes.io/projected/047ade14-fb4a-4623-a579-087c0ed54327-kube-api-access-rs28q\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598650    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598683    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22c6k\" (UniqueName: \"kubernetes.io/projected/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-kube-api-access-22c6k\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598713    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-socket-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598737    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njlhf\" (UniqueName: \"kubernetes.io/projected/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-kube-api-access-njlhf\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598760    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-modprobe-d\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598782    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-etc-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598804    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-ovn\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598844    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-kubelet\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598875    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598901    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv7h8\" (UniqueName: \"kubernetes.io/projected/60c13910-366d-437c-b0a1-e3c83a770325-kube-api-access-lv7h8\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598903    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-socket-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598923    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/186e59bb-668c-4cb7-b099-f0f86ee14eaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598950    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-slash\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598971    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-kubelet\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598976    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:18.599523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598974    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-etc-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598996    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-slash\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.598998    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599029    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599083    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e-agent-certs\") pod \"konnectivity-agent-l69wk\" (UID: \"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e\") " pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599101    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-ovn\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599194    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599238    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-modprobe-d\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599235    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599299    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599300    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-os-release\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599326    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599378    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-os-release\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599403    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovnkube-script-lib\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599443    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-log-socket\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599471    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-system-cni-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599495    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-log-socket\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.600351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599497    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-registration-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599544    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-run\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599562    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-registration-dir\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599569    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599610    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-run-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599612    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-run\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599631    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-k8s-cni-cncf-io\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599657    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-kubelet\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599686    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-tuned\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599903    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovnkube-script-lib\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.599949    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-serviceca\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600008    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-cni-netd\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600037    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-env-overrides\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600076    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-cni-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600100    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e-konnectivity-ca\") pod \"konnectivity-agent-l69wk\" (UID: \"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e\") " pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600124    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysctl-conf\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600149    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-var-lib-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600171    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-node-log\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600199    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600223    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/047ade14-fb4a-4623-a579-087c0ed54327-multus-daemon-config\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600251    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600261    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a88267a2-ec84-4eb3-a673-c6ca74aaf192-tmp\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600300    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-systemd\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600307    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-sysctl-conf\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600328    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-system-cni-dir\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600354    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7rw4\" (UniqueName: \"kubernetes.io/projected/fc717d08-6567-4c6b-9cdd-1d260730c235-kube-api-access-x7rw4\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600367    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600385    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-etc-kubernetes\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600411    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0acdcc26-44da-455e-a2df-a871c3c9a79f-dbus\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600414    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-var-lib-openvswitch\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600428    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-host-cni-netd\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600435    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-sys\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600455    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-node-log\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600388    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-ovn-node-metrics-cert\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.601959 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600515    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-sys\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.602757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600532    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-systemd\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.602757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600575    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/186e59bb-668c-4cb7-b099-f0f86ee14eaf-system-cni-dir\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.602757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.600789    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e-konnectivity-ca\") pod \"konnectivity-agent-l69wk\" (UID: \"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e\") " pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.602757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.601032    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-serviceca\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.602757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.601032    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-env-overrides\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.602757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.601735    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e-agent-certs\") pod \"konnectivity-agent-l69wk\" (UID: \"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e\") " pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.602958 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.602896    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a88267a2-ec84-4eb3-a673-c6ca74aaf192-etc-tuned\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.611022 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.610996    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh6tc\" (UniqueName: \"kubernetes.io/projected/186e59bb-668c-4cb7-b099-f0f86ee14eaf-kube-api-access-vh6tc\") pod \"multus-additional-cni-plugins-cmp9c\" (UID: \"186e59bb-668c-4cb7-b099-f0f86ee14eaf\") " pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.611347 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.611296    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcq46\" (UniqueName: \"kubernetes.io/projected/a88267a2-ec84-4eb3-a673-c6ca74aaf192-kube-api-access-kcq46\") pod \"tuned-gjvql\" (UID: \"a88267a2-ec84-4eb3-a673-c6ca74aaf192\") " pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.613461 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.613441    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2tqr\" (UniqueName: \"kubernetes.io/projected/2502b9b3-8faa-4de3-bdcf-4faa8c95ba36-kube-api-access-p2tqr\") pod \"ovnkube-node-pwjcx\" (UID: \"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.614886 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.614836    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njlhf\" (UniqueName: \"kubernetes.io/projected/81dd57de-65e6-4ba2-8efe-3fe9fd8471ed-kube-api-access-njlhf\") pod \"aws-ebs-csi-driver-node-cnth4\" (UID: \"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.615643 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.615622    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22c6k\" (UniqueName: \"kubernetes.io/projected/5dc9cd33-0fcc-4350-91b3-a9ce3df6b689-kube-api-access-22c6k\") pod \"node-ca-kv96s\" (UID: \"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689\") " pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.662599 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.662552    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal" event={"ID":"9020b496f2ad10a4002b5e25dcae18a0","Type":"ContainerStarted","Data":"de6e35d29699326101f572873fc4b2b58950069c08d1b3545cdd9c3d1c185ffe"}
Jul 10 13:26:18.663626 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.663601    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal" event={"ID":"fcfe08aec5c2bd135c282f4562048962","Type":"ContainerStarted","Data":"05b0a261db6c7736d16f29559d58c6a379a6e2aa1810f11426c99432497fd047"}
Jul 10 13:26:18.701082 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701061    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-socket-dir-parent\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701177 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701088    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60c13910-366d-437c-b0a1-e3c83a770325-host-slash\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.701177 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701113    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-cni-multus\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701177 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701129    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-conf-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701177 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701152    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/60c13910-366d-437c-b0a1-e3c83a770325-iptables-alerter-script\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701188    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-socket-dir-parent\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701202    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-cni-multus\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701188    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60c13910-366d-437c-b0a1-e3c83a770325-host-slash\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701217    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-conf-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701269    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc717d08-6567-4c6b-9cdd-1d260730c235-hosts-file\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701320    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc717d08-6567-4c6b-9cdd-1d260730c235-tmp-dir\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701346    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-cnibin\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701351    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc717d08-6567-4c6b-9cdd-1d260730c235-hosts-file\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.701372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701370    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-os-release\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701394    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/047ade14-fb4a-4623-a579-087c0ed54327-cni-binary-copy\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701419    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-hostroot\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701410    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-cnibin\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701489    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-os-release\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701525    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-hostroot\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701555    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0acdcc26-44da-455e-a2df-a871c3c9a79f-kubelet-config\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701583    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrcr\" (UniqueName: \"kubernetes.io/projected/ce6950c4-e47b-45eb-9ad1-82ae182964b1-kube-api-access-8hrcr\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701605    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-netns\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701629    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-cni-bin\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701648    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc717d08-6567-4c6b-9cdd-1d260730c235-tmp-dir\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701659    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-multus-certs\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701673    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-netns\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701711    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/60c13910-366d-437c-b0a1-e3c83a770325-iptables-alerter-script\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.701726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701716    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs28q\" (UniqueName: \"kubernetes.io/projected/047ade14-fb4a-4623-a579-087c0ed54327-kube-api-access-rs28q\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701753    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv7h8\" (UniqueName: \"kubernetes.io/projected/60c13910-366d-437c-b0a1-e3c83a770325-kube-api-access-lv7h8\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701779    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701803    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701850    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-system-cni-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701884    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-k8s-cni-cncf-io\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701909    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-cni-bin\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701916    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-kubelet\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701714    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-multus-certs\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701933    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/047ade14-fb4a-4623-a579-087c0ed54327-cni-binary-copy\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701948    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-cni-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701753    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0acdcc26-44da-455e-a2df-a871c3c9a79f-kubelet-config\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701963    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-var-lib-kubelet\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.701978    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/047ade14-fb4a-4623-a579-087c0ed54327-multus-daemon-config\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702004    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702018    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-system-cni-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702008    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-host-run-k8s-cni-cncf-io\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702031    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7rw4\" (UniqueName: \"kubernetes.io/projected/fc717d08-6567-4c6b-9cdd-1d260730c235-kube-api-access-x7rw4\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.702242 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702055    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-etc-kubernetes\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702076    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-multus-cni-dir\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702079    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0acdcc26-44da-455e-a2df-a871c3c9a79f-dbus\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.702008    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.702087    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702148    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/047ade14-fb4a-4623-a579-087c0ed54327-etc-kubernetes\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702188    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0acdcc26-44da-455e-a2df-a871c3c9a79f-dbus\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.702203    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs podName:ce6950c4-e47b-45eb-9ad1-82ae182964b1 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:19.202181552 +0000 UTC m=+3.068308273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs") pod "network-metrics-daemon-4qt4r" (UID: "ce6950c4-e47b-45eb-9ad1-82ae182964b1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.702222    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret podName:0acdcc26-44da-455e-a2df-a871c3c9a79f nodeName:}" failed. No retries permitted until 2026-07-10 13:26:19.202212366 +0000 UTC m=+3.068339000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret") pod "global-pull-secret-syncer-ggn9l" (UID: "0acdcc26-44da-455e-a2df-a871c3c9a79f") : object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:18.702967 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.702415    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/047ade14-fb4a-4623-a579-087c0ed54327-multus-daemon-config\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.721544 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.721492    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 13:26:18.721544 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.721520    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 13:26:18.721544 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.721533    2554 projected.go:196] Error preparing data for projected volume kube-api-access-bcsbw for pod openshift-network-diagnostics/network-check-target-8swsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:18.721740 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:18.721595    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw podName:f4047511-c962-4549-a800-86acd8d84803 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:19.221578178 +0000 UTC m=+3.087704844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bcsbw" (UniqueName: "kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw") pod "network-check-target-8swsb" (UID: "f4047511-c962-4549-a800-86acd8d84803") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:18.729185 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.729164    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs28q\" (UniqueName: \"kubernetes.io/projected/047ade14-fb4a-4623-a579-087c0ed54327-kube-api-access-rs28q\") pod \"multus-zjqnn\" (UID: \"047ade14-fb4a-4623-a579-087c0ed54327\") " pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.729267 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.729175    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7rw4\" (UniqueName: \"kubernetes.io/projected/fc717d08-6567-4c6b-9cdd-1d260730c235-kube-api-access-x7rw4\") pod \"node-resolver-g4sfq\" (UID: \"fc717d08-6567-4c6b-9cdd-1d260730c235\") " pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.729340 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.729324    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv7h8\" (UniqueName: \"kubernetes.io/projected/60c13910-366d-437c-b0a1-e3c83a770325-kube-api-access-lv7h8\") pod \"iptables-alerter-j5x7m\" (UID: \"60c13910-366d-437c-b0a1-e3c83a770325\") " pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.729597 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.729582    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrcr\" (UniqueName: \"kubernetes.io/projected/ce6950c4-e47b-45eb-9ad1-82ae182964b1-kube-api-access-8hrcr\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:18.790662 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.790646    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4"
Jul 10 13:26:18.798429 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.798412    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gjvql"
Jul 10 13:26:18.807209 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.807193    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kv96s"
Jul 10 13:26:18.811684 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.811669    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cmp9c"
Jul 10 13:26:18.819311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.819294    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:18.826912 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.826894    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:18.834487 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.834460    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g4sfq"
Jul 10 13:26:18.842090 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.842070    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zjqnn"
Jul 10 13:26:18.849760 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.849742    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5x7m"
Jul 10 13:26:18.886013 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:18.885993    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 13:26:19.205671 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.205632    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:19.205846 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.205707    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:19.205846 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.205786    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:19.205930 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.205857    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:19.205930 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.205861    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs podName:ce6950c4-e47b-45eb-9ad1-82ae182964b1 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:20.205844961 +0000 UTC m=+4.071971597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs") pod "network-metrics-daemon-4qt4r" (UID: "ce6950c4-e47b-45eb-9ad1-82ae182964b1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:19.205930 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.205915    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret podName:0acdcc26-44da-455e-a2df-a871c3c9a79f nodeName:}" failed. No retries permitted until 2026-07-10 13:26:20.205902763 +0000 UTC m=+4.072029395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret") pod "global-pull-secret-syncer-ggn9l" (UID: "0acdcc26-44da-455e-a2df-a871c3c9a79f") : object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:19.306625 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.306592    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:19.306759 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.306716    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 13:26:19.306759 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.306728    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 13:26:19.306759 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.306736    2554 projected.go:196] Error preparing data for projected volume kube-api-access-bcsbw for pod openshift-network-diagnostics/network-check-target-8swsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:19.306895 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.306780    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw podName:f4047511-c962-4549-a800-86acd8d84803 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:20.306768423 +0000 UTC m=+4.172895056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcsbw" (UniqueName: "kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw") pod "network-check-target-8swsb" (UID: "f4047511-c962-4549-a800-86acd8d84803") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:19.536957 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.536875    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 13:21:18 +0000 UTC" deadline="2028-04-13 18:06:45.57549654 +0000 UTC"
Jul 10 13:26:19.536957 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.536910    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15436h40m26.038591254s"
Jul 10 13:26:19.581612 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.581584    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2502b9b3_8faa_4de3_bdcf_4faa8c95ba36.slice/crio-cce27b051a1bbde40bdccba5b388967c0f4352d7db4cab3dabe49373310b82b0 WatchSource:0}: Error finding container cce27b051a1bbde40bdccba5b388967c0f4352d7db4cab3dabe49373310b82b0: Status 404 returned error can't find the container with id cce27b051a1bbde40bdccba5b388967c0f4352d7db4cab3dabe49373310b82b0
Jul 10 13:26:19.600505 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.600485    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc9cd33_0fcc_4350_91b3_a9ce3df6b689.slice/crio-95441d2956d560b1716c17b50d80a0b464c5315c99770a90dfd038fd5808aab0 WatchSource:0}: Error finding container 95441d2956d560b1716c17b50d80a0b464c5315c99770a90dfd038fd5808aab0: Status 404 returned error can't find the container with id 95441d2956d560b1716c17b50d80a0b464c5315c99770a90dfd038fd5808aab0
Jul 10 13:26:19.601476 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.601453    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88267a2_ec84_4eb3_a673_c6ca74aaf192.slice/crio-4aed61ffaaa52da350f2729b2dec9931a8c9b799d7b9c2cd409d0c722b65af96 WatchSource:0}: Error finding container 4aed61ffaaa52da350f2729b2dec9931a8c9b799d7b9c2cd409d0c722b65af96: Status 404 returned error can't find the container with id 4aed61ffaaa52da350f2729b2dec9931a8c9b799d7b9c2cd409d0c722b65af96
Jul 10 13:26:19.602379 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.602355    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a7c7869_c5b0_46ad_b31d_8dc7bfa9668e.slice/crio-1f41801a9fabbbd83d362f82e6b0675d7ec6a410614f838f4e492f6387da1937 WatchSource:0}: Error finding container 1f41801a9fabbbd83d362f82e6b0675d7ec6a410614f838f4e492f6387da1937: Status 404 returned error can't find the container with id 1f41801a9fabbbd83d362f82e6b0675d7ec6a410614f838f4e492f6387da1937
Jul 10 13:26:19.603867 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.603753    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c13910_366d_437c_b0a1_e3c83a770325.slice/crio-ab8021b4c1dd2f2e6722e86b460c3407b3c3bfb7e60111453449e207309ea929 WatchSource:0}: Error finding container ab8021b4c1dd2f2e6722e86b460c3407b3c3bfb7e60111453449e207309ea929: Status 404 returned error can't find the container with id ab8021b4c1dd2f2e6722e86b460c3407b3c3bfb7e60111453449e207309ea929
Jul 10 13:26:19.605112 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.605085    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81dd57de_65e6_4ba2_8efe_3fe9fd8471ed.slice/crio-62f11c1d30149ec78e6ee82470a417f7ea2cf785b3727bc91545bb3dc7d7dad9 WatchSource:0}: Error finding container 62f11c1d30149ec78e6ee82470a417f7ea2cf785b3727bc91545bb3dc7d7dad9: Status 404 returned error can't find the container with id 62f11c1d30149ec78e6ee82470a417f7ea2cf785b3727bc91545bb3dc7d7dad9
Jul 10 13:26:19.606991 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.606889    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186e59bb_668c_4cb7_b099_f0f86ee14eaf.slice/crio-8e1df6c595d924ab3ee80fd6204de4378adaceaf54722b14b4fe9f335ad6a140 WatchSource:0}: Error finding container 8e1df6c595d924ab3ee80fd6204de4378adaceaf54722b14b4fe9f335ad6a140: Status 404 returned error can't find the container with id 8e1df6c595d924ab3ee80fd6204de4378adaceaf54722b14b4fe9f335ad6a140
Jul 10 13:26:19.607859 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.607800    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047ade14_fb4a_4623_a579_087c0ed54327.slice/crio-f18fbd80482a3455d9ab4a0b529aa911f38708b849285d8d703e8ed63e299785 WatchSource:0}: Error finding container f18fbd80482a3455d9ab4a0b529aa911f38708b849285d8d703e8ed63e299785: Status 404 returned error can't find the container with id f18fbd80482a3455d9ab4a0b529aa911f38708b849285d8d703e8ed63e299785
Jul 10 13:26:19.609797 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:19.609773    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc717d08_6567_4c6b_9cdd_1d260730c235.slice/crio-64026624bb6c9b63238aa67c09a8408c5936ed5e5ef24f5ae3089a374c9079f1 WatchSource:0}: Error finding container 64026624bb6c9b63238aa67c09a8408c5936ed5e5ef24f5ae3089a374c9079f1: Status 404 returned error can't find the container with id 64026624bb6c9b63238aa67c09a8408c5936ed5e5ef24f5ae3089a374c9079f1
Jul 10 13:26:19.658289 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.658225    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:19.658356 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.658333    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:19.658356 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.658339    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:19.658450 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.658365    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:19.658506 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.658481    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:19.658605 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:19.658586    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:19.666544 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.666522    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4" event={"ID":"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed","Type":"ContainerStarted","Data":"62f11c1d30149ec78e6ee82470a417f7ea2cf785b3727bc91545bb3dc7d7dad9"}
Jul 10 13:26:19.667679 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.667653    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5x7m" event={"ID":"60c13910-366d-437c-b0a1-e3c83a770325","Type":"ContainerStarted","Data":"ab8021b4c1dd2f2e6722e86b460c3407b3c3bfb7e60111453449e207309ea929"}
Jul 10 13:26:19.668944 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.668923    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gjvql" event={"ID":"a88267a2-ec84-4eb3-a673-c6ca74aaf192","Type":"ContainerStarted","Data":"4aed61ffaaa52da350f2729b2dec9931a8c9b799d7b9c2cd409d0c722b65af96"}
Jul 10 13:26:19.672573 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.672542    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g4sfq" event={"ID":"fc717d08-6567-4c6b-9cdd-1d260730c235","Type":"ContainerStarted","Data":"64026624bb6c9b63238aa67c09a8408c5936ed5e5ef24f5ae3089a374c9079f1"}
Jul 10 13:26:19.673575 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.673545    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l69wk" event={"ID":"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e","Type":"ContainerStarted","Data":"1f41801a9fabbbd83d362f82e6b0675d7ec6a410614f838f4e492f6387da1937"}
Jul 10 13:26:19.674566 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.674541    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kv96s" event={"ID":"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689","Type":"ContainerStarted","Data":"95441d2956d560b1716c17b50d80a0b464c5315c99770a90dfd038fd5808aab0"}
Jul 10 13:26:19.675711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.675690    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"cce27b051a1bbde40bdccba5b388967c0f4352d7db4cab3dabe49373310b82b0"}
Jul 10 13:26:19.676799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.676779    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zjqnn" event={"ID":"047ade14-fb4a-4623-a579-087c0ed54327","Type":"ContainerStarted","Data":"f18fbd80482a3455d9ab4a0b529aa911f38708b849285d8d703e8ed63e299785"}
Jul 10 13:26:19.677806 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:19.677788    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerStarted","Data":"8e1df6c595d924ab3ee80fd6204de4378adaceaf54722b14b4fe9f335ad6a140"}
Jul 10 13:26:20.212750 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.212115    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:20.212750 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.212212    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:20.212750 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.212326    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:20.212750 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.212379    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret podName:0acdcc26-44da-455e-a2df-a871c3c9a79f nodeName:}" failed. No retries permitted until 2026-07-10 13:26:22.212362823 +0000 UTC m=+6.078489472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret") pod "global-pull-secret-syncer-ggn9l" (UID: "0acdcc26-44da-455e-a2df-a871c3c9a79f") : object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:20.212750 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.212744    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:20.213142 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.212792    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs podName:ce6950c4-e47b-45eb-9ad1-82ae182964b1 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:22.212777949 +0000 UTC m=+6.078904581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs") pod "network-metrics-daemon-4qt4r" (UID: "ce6950c4-e47b-45eb-9ad1-82ae182964b1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:20.295798 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.295615    2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 13:26:20.313289 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.312777    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:20.313289 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.312929    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 13:26:20.313289 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.312952    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 13:26:20.313289 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.312964    2554 projected.go:196] Error preparing data for projected volume kube-api-access-bcsbw for pod openshift-network-diagnostics/network-check-target-8swsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:20.313289 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:20.313017    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw podName:f4047511-c962-4549-a800-86acd8d84803 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:22.312999532 +0000 UTC m=+6.179126178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcsbw" (UniqueName: "kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw") pod "network-check-target-8swsb" (UID: "f4047511-c962-4549-a800-86acd8d84803") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:20.537717 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.537612    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 13:21:18 +0000 UTC" deadline="2028-03-30 19:56:18.658699278 +0000 UTC"
Jul 10 13:26:20.537717 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.537672    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15102h29m58.121031541s"
Jul 10 13:26:20.688273 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.688233    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal" event={"ID":"fcfe08aec5c2bd135c282f4562048962","Type":"ContainerStarted","Data":"cdb0f03f416d6de0b78c339e4d1546dfbade9e5b3b81625faf337959ca9c0b55"}
Jul 10 13:26:20.700975 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.700948    2554 generic.go:356] "Generic (PLEG): container finished" podID="9020b496f2ad10a4002b5e25dcae18a0" containerID="c7b6c13eeef1a1b025e6c3d3ec26d61a1f65614e2ba3f6037ddfb29c99920afd" exitCode=0
Jul 10 13:26:20.701099 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.700990    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal" event={"ID":"9020b496f2ad10a4002b5e25dcae18a0","Type":"ContainerDied","Data":"c7b6c13eeef1a1b025e6c3d3ec26d61a1f65614e2ba3f6037ddfb29c99920afd"}
Jul 10 13:26:20.740713 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:20.740161    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-59.ec2.internal" podStartSLOduration=2.740135802 podStartE2EDuration="2.740135802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:20.724041872 +0000 UTC m=+4.590168551" watchObservedRunningTime="2026-07-10 13:26:20.740135802 +0000 UTC m=+4.606262457"
Jul 10 13:26:21.659059 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:21.658278    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:21.659059 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:21.658411    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:21.659059 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:21.658771    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:21.659059 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:21.658871    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:21.659059 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:21.658948    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:21.659059 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:21.659017    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:21.717988 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:21.717912    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal" event={"ID":"9020b496f2ad10a4002b5e25dcae18a0","Type":"ContainerStarted","Data":"a8fea8fc5d64f72d304a1ff4b33a82f82e8fe088d0bb71211343316ec0549270"}
Jul 10 13:26:22.229248 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:22.229214    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:22.229439 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:22.229293    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:22.229439 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.229419    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:22.229539 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.229479    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs podName:ce6950c4-e47b-45eb-9ad1-82ae182964b1 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:26.229460937 +0000 UTC m=+10.095587583 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs") pod "network-metrics-daemon-4qt4r" (UID: "ce6950c4-e47b-45eb-9ad1-82ae182964b1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:22.229745 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.229611    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:22.229745 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.229677    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret podName:0acdcc26-44da-455e-a2df-a871c3c9a79f nodeName:}" failed. No retries permitted until 2026-07-10 13:26:26.229659724 +0000 UTC m=+10.095786372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret") pod "global-pull-secret-syncer-ggn9l" (UID: "0acdcc26-44da-455e-a2df-a871c3c9a79f") : object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:22.330463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:22.329839    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:22.330463 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.329998    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 13:26:22.330463 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.330015    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 13:26:22.330463 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.330027    2554 projected.go:196] Error preparing data for projected volume kube-api-access-bcsbw for pod openshift-network-diagnostics/network-check-target-8swsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:22.330463 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:22.330091    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw podName:f4047511-c962-4549-a800-86acd8d84803 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:26.330072622 +0000 UTC m=+10.196199269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcsbw" (UniqueName: "kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw") pod "network-check-target-8swsb" (UID: "f4047511-c962-4549-a800-86acd8d84803") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:23.658531 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:23.658499    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:23.659001 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:23.658499    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:23.659001 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:23.658636    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:23.659001 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:23.658694    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:23.659001 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:23.658496    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:23.659001 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:23.658783    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:25.658642 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:25.658495    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:25.659092 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:25.658641    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:25.659092 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:25.658679    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:25.659092 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:25.658794    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:25.659092 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:25.658859    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:25.659092 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:25.658971    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:26.260793 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:26.260759    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:26.260989 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:26.260872    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:26.260989 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.260951    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:26.261096 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.261008    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:26.261096 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.261052    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret podName:0acdcc26-44da-455e-a2df-a871c3c9a79f nodeName:}" failed. No retries permitted until 2026-07-10 13:26:34.261031293 +0000 UTC m=+18.127157937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret") pod "global-pull-secret-syncer-ggn9l" (UID: "0acdcc26-44da-455e-a2df-a871c3c9a79f") : object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:26.261096 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.261073    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs podName:ce6950c4-e47b-45eb-9ad1-82ae182964b1 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:34.261062636 +0000 UTC m=+18.127189275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs") pod "network-metrics-daemon-4qt4r" (UID: "ce6950c4-e47b-45eb-9ad1-82ae182964b1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:26.361515 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:26.361422    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:26.361665 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.361607    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 13:26:26.361665 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.361627    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 13:26:26.361665 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.361639    2554 projected.go:196] Error preparing data for projected volume kube-api-access-bcsbw for pod openshift-network-diagnostics/network-check-target-8swsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:26.361806 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:26.361696    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw podName:f4047511-c962-4549-a800-86acd8d84803 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:34.361679063 +0000 UTC m=+18.227805710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcsbw" (UniqueName: "kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw") pod "network-check-target-8swsb" (UID: "f4047511-c962-4549-a800-86acd8d84803") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:27.659218 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:27.658800    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:27.659218 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:27.658849    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:27.659218 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:27.658849    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:27.659218 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:27.658935    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:27.659218 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:27.659048    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:27.659218 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:27.659141    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:29.657968 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:29.657940    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:29.657968 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:29.657963    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:29.658389 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:29.657942    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:29.658389 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:29.658038    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:29.658389 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:29.658119    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:29.658389 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:29.658200    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:31.658856 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:31.658799    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:31.659277 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:31.658860    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:31.659277 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:31.658865    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:31.659277 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:31.658954    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:31.659277 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:31.659018    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:31.659277 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:31.659091    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:33.658110 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:33.658076    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:33.658551 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:33.658078    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:33.658551 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:33.658218    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:33.658551 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:33.658076    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:33.658551 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:33.658279    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:33.658551 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:33.658368    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:34.320186 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:34.320152    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:34.320332 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:34.320221    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:34.320332 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.320302    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:34.320438 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.320323    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:34.320438 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.320372    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs podName:ce6950c4-e47b-45eb-9ad1-82ae182964b1 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:50.320351731 +0000 UTC m=+34.186478379 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs") pod "network-metrics-daemon-4qt4r" (UID: "ce6950c4-e47b-45eb-9ad1-82ae182964b1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:34.320438 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.320397    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret podName:0acdcc26-44da-455e-a2df-a871c3c9a79f nodeName:}" failed. No retries permitted until 2026-07-10 13:26:50.320384416 +0000 UTC m=+34.186511061 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret") pod "global-pull-secret-syncer-ggn9l" (UID: "0acdcc26-44da-455e-a2df-a871c3c9a79f") : object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:34.421507 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:34.421428    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:34.421650 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.421601    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 13:26:34.421650 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.421627    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 13:26:34.421650 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.421637    2554 projected.go:196] Error preparing data for projected volume kube-api-access-bcsbw for pod openshift-network-diagnostics/network-check-target-8swsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:34.421772 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:34.421689    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw podName:f4047511-c962-4549-a800-86acd8d84803 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:50.421672612 +0000 UTC m=+34.287799268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcsbw" (UniqueName: "kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw") pod "network-check-target-8swsb" (UID: "f4047511-c962-4549-a800-86acd8d84803") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:35.657994 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:35.657761    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:35.658420 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:35.657814    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:35.658420 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:35.658074    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:35.658420 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:35.658145    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:35.658420 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:35.657837    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:35.658420 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:35.658280    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:37.658768 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:37.658733    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:37.658768 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:37.658767    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:37.659290 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:37.658777    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:37.659290 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:37.658853    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:37.659290 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:37.658928    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:37.659290 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:37.659025    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:38.751278 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.750950    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zjqnn" event={"ID":"047ade14-fb4a-4623-a579-087c0ed54327","Type":"ContainerStarted","Data":"034304ba55990c677d34a563c609eed031d9cab3828bec617fcd2cc91740d3f5"}
Jul 10 13:26:38.753406 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.753120    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerStarted","Data":"302944dd0b0aa333d541da26d205ab45be061f2fbb13c85fdbfa259cae8f6fdb"}
Jul 10 13:26:38.756284 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.756253    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4" event={"ID":"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed","Type":"ContainerStarted","Data":"d03734e8009142e21b2307afcb28299e150eb019f127f9c6c3af9ebc2769aa3b"}
Jul 10 13:26:38.758109 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.758077    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gjvql" event={"ID":"a88267a2-ec84-4eb3-a673-c6ca74aaf192","Type":"ContainerStarted","Data":"43dbdd5b52e41ec61ac03efce83e68b32f6549866e53b3edc3317c0fd06d5d1f"}
Jul 10 13:26:38.759808 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.759782    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g4sfq" event={"ID":"fc717d08-6567-4c6b-9cdd-1d260730c235","Type":"ContainerStarted","Data":"b0bb7d6d5f0101d7f1de2341a0acb8847385c5606da7df5063d92e2fcd36d8df"}
Jul 10 13:26:38.762253 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.762219    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l69wk" event={"ID":"3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e","Type":"ContainerStarted","Data":"688a0160b2bfcf3cd95a0c1a23458679e8135ff3326c2a5607f3c4d883434a02"}
Jul 10 13:26:38.764201 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.764164    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kv96s" event={"ID":"5dc9cd33-0fcc-4350-91b3-a9ce3df6b689","Type":"ContainerStarted","Data":"193168804f64a5dda178342179964cb00912e86ea3b8b74a1ade02c72819a832"}
Jul 10 13:26:38.766026 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.766004    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"de3bce953eb1242e372f6d05f6b6d4dce044d022654fb6fe5f4d3c38d961ba44"}
Jul 10 13:26:38.783812 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.783758    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zjqnn" podStartSLOduration=3.858277211 podStartE2EDuration="22.783740031s" totalImagesPullingTime="18.92546282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:38.783011849 +0000 UTC m=+22.649138497" watchObservedRunningTime="2026-07-10 13:26:38.783740031 +0000 UTC m=+22.649866687"
Jul 10 13:26:38.784149 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.784113    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-59.ec2.internal" podStartSLOduration=20.784102855 podStartE2EDuration="20.784102855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:21.736542528 +0000 UTC m=+5.602669184" watchObservedRunningTime="2026-07-10 13:26:38.784102855 +0000 UTC m=+22.650229509"
Jul 10 13:26:38.803415 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.803360    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kv96s" podStartSLOduration=3.9050152320000002 podStartE2EDuration="22.8033419s" totalImagesPullingTime="18.898326668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:38.803066614 +0000 UTC m=+22.669193268" watchObservedRunningTime="2026-07-10 13:26:38.8033419 +0000 UTC m=+22.669468561"
Jul 10 13:26:38.840988 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.840940    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l69wk" podStartSLOduration=3.944804804 podStartE2EDuration="22.840922472s" totalImagesPullingTime="18.896117668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:38.840241311 +0000 UTC m=+22.706367965" watchObservedRunningTime="2026-07-10 13:26:38.840922472 +0000 UTC m=+22.707049127"
Jul 10 13:26:38.858358 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.858308    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g4sfq" podStartSLOduration=3.970426928 podStartE2EDuration="22.858288406s" totalImagesPullingTime="18.887861478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:38.857430314 +0000 UTC m=+22.723556986" watchObservedRunningTime="2026-07-10 13:26:38.858288406 +0000 UTC m=+22.724415073"
Jul 10 13:26:38.876608 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:38.876573    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gjvql" podStartSLOduration=3.979654672 podStartE2EDuration="22.876559015s" totalImagesPullingTime="18.896904343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:38.876204409 +0000 UTC m=+22.742331064" watchObservedRunningTime="2026-07-10 13:26:38.876559015 +0000 UTC m=+22.742685669"
Jul 10 13:26:39.658054 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.658025    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:39.658257 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:39.658122    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:39.658257 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.658130    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:39.658257 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.658125    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:39.658257 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:39.658217    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:39.658257 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:39.658247    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:39.684175 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.684152    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:39.684639 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.684624    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:39.770596 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.770573    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"a931c3c0a5d41fe26a0bd02c6c8ca6096cff486d73dcd0ec6a8894353a0edac4"}
Jul 10 13:26:39.770931 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.770649    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"90a2805359115e98885ce3aaf53bdc7277640c3b34bc53893a1f7bcd2365473d"}
Jul 10 13:26:39.770931 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.770698    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"a6c2e072fa9ea3830a9918f746a55ee0eb448f1ae6622bb4af0e4fff68e62acf"}
Jul 10 13:26:39.770931 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.770745    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"5a1742fafc8ecf3702c0d37f674ce3da2c67048de92881e2f6386cac8ebe072e"}
Jul 10 13:26:39.770931 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.770794    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"7c604cdb13daf0abe76c3c8a5401075141675f4270647343fc5cc81f50d79bb1"}
Jul 10 13:26:39.775127 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.775104    2554 generic.go:356] "Generic (PLEG): container finished" podID="186e59bb-668c-4cb7-b099-f0f86ee14eaf" containerID="302944dd0b0aa333d541da26d205ab45be061f2fbb13c85fdbfa259cae8f6fdb" exitCode=0
Jul 10 13:26:39.775220 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:39.775200    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerDied","Data":"302944dd0b0aa333d541da26d205ab45be061f2fbb13c85fdbfa259cae8f6fdb"}
Jul 10 13:26:40.002991 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:40.002852    2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 13:26:40.557099 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:40.557012    2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T13:26:40.002987885Z","UUID":"51ed41f8-975d-460d-ade6-548f8b757f31","Handler":null,"Name":"","Endpoint":""}
Jul 10 13:26:40.558485 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:40.558469    2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 13:26:40.558591 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:40.558499    2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 13:26:40.779969 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:40.779939    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4" event={"ID":"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed","Type":"ContainerStarted","Data":"b9215f9f8a4ed3ff53b6ad7258488d892677845155a51b13c71a4c9f2daf94bf"}
Jul 10 13:26:40.781487 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:40.781455    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5x7m" event={"ID":"60c13910-366d-437c-b0a1-e3c83a770325","Type":"ContainerStarted","Data":"8d18d8e92df166e163214dfcca3221244fc41b7bdae49c1bc396dc43c7f4791a"}
Jul 10 13:26:40.781487 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:40.781473    2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 13:26:41.658482 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:41.658459    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:41.658652 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:41.658498    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:41.658652 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:41.658556    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:41.658652 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:41.658571    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:41.658652 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:41.658635    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:41.658880 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:41.658694    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:41.785594 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:41.785515    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4" event={"ID":"81dd57de-65e6-4ba2-8efe-3fe9fd8471ed","Type":"ContainerStarted","Data":"c1e4e92e84853c807fd259f3f889ca3364b789a8194833f33ba918485a75de39"}
Jul 10 13:26:41.788670 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:41.788648    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"6b6a2644a8cee9caf50c5e1f2c95d6e2a3814c070fef02415e54113c49489c8e"}
Jul 10 13:26:41.806017 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:41.805980    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnth4" podStartSLOduration=4.3314400840000005 podStartE2EDuration="25.805966554s" totalImagesPullingTime="21.47452647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:41.804907902 +0000 UTC m=+25.671034556" watchObservedRunningTime="2026-07-10 13:26:41.805966554 +0000 UTC m=+25.672093208"
Jul 10 13:26:41.808886 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:41.807528    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j5x7m" podStartSLOduration=6.913823201 podStartE2EDuration="25.807510414s" totalImagesPullingTime="18.893687213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:40.797944147 +0000 UTC m=+24.664070800" watchObservedRunningTime="2026-07-10 13:26:41.807510414 +0000 UTC m=+25.673637070"
Jul 10 13:26:43.658362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:43.658333    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:43.659007 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:43.658371    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:43.659007 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:43.658371    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:43.659007 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:43.658456    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:43.659007 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:43.658555    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:43.659007 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:43.658601    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:45.056018 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.055725    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:45.056763 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.056110    2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 13:26:45.056763 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.056531    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l69wk"
Jul 10 13:26:45.658078 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.658027    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:45.658179 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.658027    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:45.658179 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:45.658134    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:45.658251 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.658027    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:45.658251 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:45.658206    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:45.658314 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:45.658284    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:45.801397 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.801369    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" event={"ID":"2502b9b3-8faa-4de3-bdcf-4faa8c95ba36","Type":"ContainerStarted","Data":"9fd4081c82c1a489dfc80332a42f1153456f41b7f65f568d42be6eea340f85bc"}
Jul 10 13:26:45.801679 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.801645    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:45.801786 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.801738    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:45.804037 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.804008    2554 generic.go:356] "Generic (PLEG): container finished" podID="186e59bb-668c-4cb7-b099-f0f86ee14eaf" containerID="b76aaf04161fb002ee600cb7a7b3369b1ea4bde229de49b658c0c47516a22f62" exitCode=0
Jul 10 13:26:45.804135 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.804105    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerDied","Data":"b76aaf04161fb002ee600cb7a7b3369b1ea4bde229de49b658c0c47516a22f62"}
Jul 10 13:26:45.820150 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.820128    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:45.820409 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.820390    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:45.836302 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:45.836261    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx" podStartSLOduration=10.823221742 podStartE2EDuration="29.836248769s" totalImagesPullingTime="19.013027027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:45.834669457 +0000 UTC m=+29.700796111" watchObservedRunningTime="2026-07-10 13:26:45.836248769 +0000 UTC m=+29.702375422"
Jul 10 13:26:46.806699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:46.806675    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:26:46.952703 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:46.952678    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ggn9l"]
Jul 10 13:26:46.952793 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:46.952783    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:46.952912 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:46.952895    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:46.955122 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:46.955100    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4qt4r"]
Jul 10 13:26:46.955227 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:46.955190    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:46.955292 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:46.955263    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:46.966898 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:46.966876    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8swsb"]
Jul 10 13:26:46.967000 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:46.966988    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:46.967086 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:46.967069    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:47.810628 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:47.810594    2554 generic.go:356] "Generic (PLEG): container finished" podID="186e59bb-668c-4cb7-b099-f0f86ee14eaf" containerID="6bde7c71310b581cbf538ab9b4ceaa41093220d0a4c8340d7d10fd9919eda093" exitCode=0
Jul 10 13:26:47.811088 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:47.810681    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerDied","Data":"6bde7c71310b581cbf538ab9b4ceaa41093220d0a4c8340d7d10fd9919eda093"}
Jul 10 13:26:48.658431 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:48.658405    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:48.658431 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:48.658424    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:48.658627 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:48.658486    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:48.658627 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:48.658496    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:48.658627 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:48.658571    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:48.658627 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:48.658614    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:49.816594 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:49.816560    2554 generic.go:356] "Generic (PLEG): container finished" podID="186e59bb-668c-4cb7-b099-f0f86ee14eaf" containerID="f7463f7773ac159c694ebd04a234d3f49d33f4792cb1e7e234592f1ea2f7415b" exitCode=0
Jul 10 13:26:49.817325 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:49.816607    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerDied","Data":"f7463f7773ac159c694ebd04a234d3f49d33f4792cb1e7e234592f1ea2f7415b"}
Jul 10 13:26:50.339145 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:50.339100    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:50.339296 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:50.339195    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:50.339296 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.339282    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:50.339411 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.339303    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:50.339411 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.339354    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs podName:ce6950c4-e47b-45eb-9ad1-82ae182964b1 nodeName:}" failed. No retries permitted until 2026-07-10 13:27:22.33933477 +0000 UTC m=+66.205461417 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs") pod "network-metrics-daemon-4qt4r" (UID: "ce6950c4-e47b-45eb-9ad1-82ae182964b1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 13:26:50.339411 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.339383    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret podName:0acdcc26-44da-455e-a2df-a871c3c9a79f nodeName:}" failed. No retries permitted until 2026-07-10 13:27:22.339364151 +0000 UTC m=+66.205490790 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret") pod "global-pull-secret-syncer-ggn9l" (UID: "0acdcc26-44da-455e-a2df-a871c3c9a79f") : object "kube-system"/"original-pull-secret" not registered
Jul 10 13:26:50.440009 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:50.439978    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:50.440162 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.440143    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 13:26:50.440215 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.440174    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 13:26:50.440215 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.440187    2554 projected.go:196] Error preparing data for projected volume kube-api-access-bcsbw for pod openshift-network-diagnostics/network-check-target-8swsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:50.440301 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.440242    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw podName:f4047511-c962-4549-a800-86acd8d84803 nodeName:}" failed. No retries permitted until 2026-07-10 13:27:22.440225081 +0000 UTC m=+66.306351715 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcsbw" (UniqueName: "kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw") pod "network-check-target-8swsb" (UID: "f4047511-c962-4549-a800-86acd8d84803") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 13:26:50.657967 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:50.657887    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:50.657967 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:50.657901    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:50.658127 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:50.658008    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:50.658127 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.658006    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8swsb" podUID="f4047511-c962-4549-a800-86acd8d84803"
Jul 10 13:26:50.658204 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.658134    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4qt4r" podUID="ce6950c4-e47b-45eb-9ad1-82ae182964b1"
Jul 10 13:26:50.658249 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:50.658214    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ggn9l" podUID="0acdcc26-44da-455e-a2df-a871c3c9a79f"
Jul 10 13:26:52.490047 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.490020    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-59.ec2.internal" event="NodeReady"
Jul 10 13:26:52.490480 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.490136    2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 13:26:52.531187 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.531162    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5ddb849bbf-2wq92"]
Jul 10 13:26:52.533127 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.533109    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.536180 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.535962    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 13:26:52.536180 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.536014    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 13:26:52.536180 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.536067    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 13:26:52.536180 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.535962    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r7g28\""
Jul 10 13:26:52.545055 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.544941    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 13:26:52.546331 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.546311    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hjb42"]
Jul 10 13:26:52.548394 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.548375    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.548921 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.548890    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ddb849bbf-2wq92"]
Jul 10 13:26:52.550899 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.550875    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 13:26:52.550991 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.550955    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 13:26:52.550991 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.550964    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 13:26:52.551119 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.551101    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wjzj9\""
Jul 10 13:26:52.551627 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.551321    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 13:26:52.555706 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.555682    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wnk2p"]
Jul 10 13:26:52.557949 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.557931    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.560492 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.560476    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 13:26:52.560572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.560521    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-whvbg\""
Jul 10 13:26:52.560572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.560543    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 13:26:52.560695 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.560590    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 13:26:52.563627 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.563609    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hjb42"]
Jul 10 13:26:52.577112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.577088    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnk2p"]
Jul 10 13:26:52.647572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.647549    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-brqtm"]
Jul 10 13:26:52.649516 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.649499    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.652079 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.652060    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 13:26:52.652323 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.652289    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sj6cx\""
Jul 10 13:26:52.652484 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.652320    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 13:26:52.658008 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.657987    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m5z9\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-kube-api-access-8m5z9\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658102 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658012    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:26:52.658102 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658023    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.658102 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658054    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-image-registry-private-configuration\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658102 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658086    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-trusted-ca\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658314 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658118    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-installation-pull-secrets\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658314 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658144    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-data-volume\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.658314 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658161    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:26:52.658314 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658247    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97483560-993f-44e7-8850-341815421e10-ca-trust-extracted\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658504 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658276    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szcbz\" (UniqueName: \"kubernetes.io/projected/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-api-access-szcbz\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.658504 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658384    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.658504 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658467    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-crio-socket\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.658649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658516    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025769c9-3932-4cd0-97cb-96e766b3157c-cert\") pod \"ingress-canary-wnk2p\" (UID: \"025769c9-3932-4cd0-97cb-96e766b3157c\") " pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.658649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658537    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658565    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-registry-certificates\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658589    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:26:52.658649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658618    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-bound-sa-token\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.658932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.658679    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j984s\" (UniqueName: \"kubernetes.io/projected/025769c9-3932-4cd0-97cb-96e766b3157c-kube-api-access-j984s\") pod \"ingress-canary-wnk2p\" (UID: \"025769c9-3932-4cd0-97cb-96e766b3157c\") " pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.661602 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.661432    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 13:26:52.661602 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.661485    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 13:26:52.661602 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.661538    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 13:26:52.661602 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.661437    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 13:26:52.661848 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.661682    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lkjb8\""
Jul 10 13:26:52.661848 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.661768    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gcrn\""
Jul 10 13:26:52.662778 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.662758    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brqtm"]
Jul 10 13:26:52.743043 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.742997    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g4sfq_fc717d08-6567-4c6b-9cdd-1d260730c235/dns-node-resolver/0.log"
Jul 10 13:26:52.759256 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759235    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-installation-pull-secrets\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.759360 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759261    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-data-volume\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.759360 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759306    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97483560-993f-44e7-8850-341815421e10-ca-trust-extracted\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.759360 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759332    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szcbz\" (UniqueName: \"kubernetes.io/projected/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-api-access-szcbz\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.759514 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759361    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.759514 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759400    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-crio-socket\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.759514 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759424    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.759514 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759474    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025769c9-3932-4cd0-97cb-96e766b3157c-cert\") pod \"ingress-canary-wnk2p\" (UID: \"025769c9-3932-4cd0-97cb-96e766b3157c\") " pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.759514 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759505    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759530    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2a7807f-1566-4212-8e13-1e82c45eca50-tmp-dir\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759558    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-registry-certificates\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759598    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-bound-sa-token\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759617    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-data-volume\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759627    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j984s\" (UniqueName: \"kubernetes.io/projected/025769c9-3932-4cd0-97cb-96e766b3157c-kube-api-access-j984s\") pod \"ingress-canary-wnk2p\" (UID: \"025769c9-3932-4cd0-97cb-96e766b3157c\") " pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759651    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a7807f-1566-4212-8e13-1e82c45eca50-config-volume\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759673    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrlh6\" (UniqueName: \"kubernetes.io/projected/f2a7807f-1566-4212-8e13-1e82c45eca50-kube-api-access-hrlh6\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759692    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97483560-993f-44e7-8850-341815421e10-ca-trust-extracted\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759701    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m5z9\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-kube-api-access-8m5z9\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.759716    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 13:26:52.759744 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759723    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759751    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-image-registry-private-configuration\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.759783    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:26:53.259765329 +0000 UTC m=+37.125891985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : configmap "kube-rbac-proxy" not found
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759850    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-trusted-ca\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.759804    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-crio-socket\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.760032    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.760122    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb849bbf-2wq92: secret "image-registry-tls" not found
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.760096    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.760177    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls podName:97483560-993f-44e7-8850-341815421e10 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:53.260160702 +0000 UTC m=+37.126287334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls") pod "image-registry-5ddb849bbf-2wq92" (UID: "97483560-993f-44e7-8850-341815421e10") : secret "image-registry-tls" not found
Jul 10 13:26:52.760304 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.760254    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:26:53.26023669 +0000 UTC m=+37.126363323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:52.760747 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.760598    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-registry-certificates\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.761051 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.761021    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-trusted-ca\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.764283 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.764264    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-installation-pull-secrets\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.764469 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.764450    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025769c9-3932-4cd0-97cb-96e766b3157c-cert\") pod \"ingress-canary-wnk2p\" (UID: \"025769c9-3932-4cd0-97cb-96e766b3157c\") " pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.764592 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.764572    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-image-registry-private-configuration\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.771694 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.771670    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szcbz\" (UniqueName: \"kubernetes.io/projected/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-api-access-szcbz\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:52.771792 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.771738    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j984s\" (UniqueName: \"kubernetes.io/projected/025769c9-3932-4cd0-97cb-96e766b3157c-kube-api-access-j984s\") pod \"ingress-canary-wnk2p\" (UID: \"025769c9-3932-4cd0-97cb-96e766b3157c\") " pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.772077 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.772054    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m5z9\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-kube-api-access-8m5z9\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.772151 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.772138    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-bound-sa-token\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:52.861257 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.861240    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.861349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.861286    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2a7807f-1566-4212-8e13-1e82c45eca50-tmp-dir\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.861391 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.861364    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 13:26:52.861391 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.861384    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a7807f-1566-4212-8e13-1e82c45eca50-config-volume\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.861452 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:52.861408    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls podName:f2a7807f-1566-4212-8e13-1e82c45eca50 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:53.36139665 +0000 UTC m=+37.227523305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls") pod "dns-default-brqtm" (UID: "f2a7807f-1566-4212-8e13-1e82c45eca50") : secret "dns-default-metrics-tls" not found
Jul 10 13:26:52.861614 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.861598    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrlh6\" (UniqueName: \"kubernetes.io/projected/f2a7807f-1566-4212-8e13-1e82c45eca50-kube-api-access-hrlh6\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.864416 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.861893    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2a7807f-1566-4212-8e13-1e82c45eca50-tmp-dir\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.864416 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.862186    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2a7807f-1566-4212-8e13-1e82c45eca50-config-volume\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:52.867126 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.867103    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wnk2p"
Jul 10 13:26:52.871740 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:52.871722    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrlh6\" (UniqueName: \"kubernetes.io/projected/f2a7807f-1566-4212-8e13-1e82c45eca50-kube-api-access-hrlh6\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:53.006618 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.006533    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wnk2p"]
Jul 10 13:26:53.016038 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:53.016012    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025769c9_3932_4cd0_97cb_96e766b3157c.slice/crio-01160bdfa52e7b65b58b4a689b5becc49fd3ffe33efc43248bca3d819028d22c WatchSource:0}: Error finding container 01160bdfa52e7b65b58b4a689b5becc49fd3ffe33efc43248bca3d819028d22c: Status 404 returned error can't find the container with id 01160bdfa52e7b65b58b4a689b5becc49fd3ffe33efc43248bca3d819028d22c
Jul 10 13:26:53.265557 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.265480    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:53.265557 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.265526    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:53.265768 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.265599    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:53.265768 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.265636    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:53.265768 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.265686    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:26:54.265669301 +0000 UTC m=+38.131795958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:53.265768 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.265758    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 13:26:53.265973 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.265775    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb849bbf-2wq92: secret "image-registry-tls" not found
Jul 10 13:26:53.265973 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.265852    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls podName:97483560-993f-44e7-8850-341815421e10 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:54.265834573 +0000 UTC m=+38.131961217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls") pod "image-registry-5ddb849bbf-2wq92" (UID: "97483560-993f-44e7-8850-341815421e10") : secret "image-registry-tls" not found
Jul 10 13:26:53.265973 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.265884    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 13:26:53.265973 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.265942    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:26:54.265927024 +0000 UTC m=+38.132053676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : configmap "kube-rbac-proxy" not found
Jul 10 13:26:53.366168 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.366127    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:53.366292 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.366267    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 13:26:53.366346 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:53.366317    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls podName:f2a7807f-1566-4212-8e13-1e82c45eca50 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:54.366304032 +0000 UTC m=+38.232430682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls") pod "dns-default-brqtm" (UID: "f2a7807f-1566-4212-8e13-1e82c45eca50") : secret "dns-default-metrics-tls" not found
Jul 10 13:26:53.720351 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.720321    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kv96s_5dc9cd33-0fcc-4350-91b3-a9ce3df6b689/node-ca/0.log"
Jul 10 13:26:53.770806 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.770723    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-22d58"]
Jul 10 13:26:53.772960 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.772881    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.777843 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.777590    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 10 13:26:53.778022 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.778002    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 10 13:26:53.779071 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.778901    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 10 13:26:53.779168 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.779138    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 10 13:26:53.780307 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.780244    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9j29k\""
Jul 10 13:26:53.798476 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.798421    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-22d58"]
Jul 10 13:26:53.827631 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.827609    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnk2p" event={"ID":"025769c9-3932-4cd0-97cb-96e766b3157c","Type":"ContainerStarted","Data":"01160bdfa52e7b65b58b4a689b5becc49fd3ffe33efc43248bca3d819028d22c"}
Jul 10 13:26:53.870243 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.870219    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69f2fb4d-4440-4815-afd9-f1d7186dcafd-signing-cabundle\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.870352 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.870253    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp69d\" (UniqueName: \"kubernetes.io/projected/69f2fb4d-4440-4815-afd9-f1d7186dcafd-kube-api-access-gp69d\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.870440 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.870412    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69f2fb4d-4440-4815-afd9-f1d7186dcafd-signing-key\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.971555 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.971520    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69f2fb4d-4440-4815-afd9-f1d7186dcafd-signing-key\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.971697 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.971588    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69f2fb4d-4440-4815-afd9-f1d7186dcafd-signing-cabundle\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.971697 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.971614    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp69d\" (UniqueName: \"kubernetes.io/projected/69f2fb4d-4440-4815-afd9-f1d7186dcafd-kube-api-access-gp69d\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.972510 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.972460    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/69f2fb4d-4440-4815-afd9-f1d7186dcafd-signing-cabundle\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.975566 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.975541    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/69f2fb4d-4440-4815-afd9-f1d7186dcafd-signing-key\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:53.982105 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:53.982081    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp69d\" (UniqueName: \"kubernetes.io/projected/69f2fb4d-4440-4815-afd9-f1d7186dcafd-kube-api-access-gp69d\") pod \"service-ca-79f68b9f4f-22d58\" (UID: \"69f2fb4d-4440-4815-afd9-f1d7186dcafd\") " pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:54.085052 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:54.084986    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-22d58"
Jul 10 13:26:54.274370 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:54.274321    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:54.274551 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:54.274410    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:54.274551 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.274482    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 13:26:54.274551 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.274496    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:54.274551 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:54.274487    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:54.274551 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.274551    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:26:56.274532659 +0000 UTC m=+40.140659307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : configmap "kube-rbac-proxy" not found
Jul 10 13:26:54.274802 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.274564    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 13:26:54.274802 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.274572    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:26:56.274562883 +0000 UTC m=+40.140689519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:54.274802 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.274577    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb849bbf-2wq92: secret "image-registry-tls" not found
Jul 10 13:26:54.274802 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.274611    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls podName:97483560-993f-44e7-8850-341815421e10 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:56.274600798 +0000 UTC m=+40.140727450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls") pod "image-registry-5ddb849bbf-2wq92" (UID: "97483560-993f-44e7-8850-341815421e10") : secret "image-registry-tls" not found
Jul 10 13:26:54.375438 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:54.375361    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:54.375565 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.375516    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 13:26:54.375626 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:54.375582    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls podName:f2a7807f-1566-4212-8e13-1e82c45eca50 nodeName:}" failed. No retries permitted until 2026-07-10 13:26:56.375564487 +0000 UTC m=+40.241691133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls") pod "dns-default-brqtm" (UID: "f2a7807f-1566-4212-8e13-1e82c45eca50") : secret "dns-default-metrics-tls" not found
Jul 10 13:26:56.292055 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.292021    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.292072    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.292147    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.292169    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.292194    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:27:00.292182004 +0000 UTC m=+44.158308636 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : configmap "kube-rbac-proxy" not found
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.292242    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.292256    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb849bbf-2wq92: secret "image-registry-tls" not found
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.292269    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.292304    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls podName:97483560-993f-44e7-8850-341815421e10 nodeName:}" failed. No retries permitted until 2026-07-10 13:27:00.292290525 +0000 UTC m=+44.158417180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls") pod "image-registry-5ddb849bbf-2wq92" (UID: "97483560-993f-44e7-8850-341815421e10") : secret "image-registry-tls" not found
Jul 10 13:26:56.292401 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.292318    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:27:00.292312229 +0000 UTC m=+44.158438861 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : secret "insights-runtime-extractor-tls" not found
Jul 10 13:26:56.393527 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.393359    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:26:56.393688 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.393673    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 13:26:56.393785 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:26:56.393733    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls podName:f2a7807f-1566-4212-8e13-1e82c45eca50 nodeName:}" failed. No retries permitted until 2026-07-10 13:27:00.393714137 +0000 UTC m=+44.259840782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls") pod "dns-default-brqtm" (UID: "f2a7807f-1566-4212-8e13-1e82c45eca50") : secret "dns-default-metrics-tls" not found
Jul 10 13:26:56.526696 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.526419    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-22d58"]
Jul 10 13:26:56.531076 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:26:56.531054    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f2fb4d_4440_4815_afd9_f1d7186dcafd.slice/crio-2d64589ae67cae4a1aef5174e349921a0091f365c9e00d84fa384c08a6b55486 WatchSource:0}: Error finding container 2d64589ae67cae4a1aef5174e349921a0091f365c9e00d84fa384c08a6b55486: Status 404 returned error can't find the container with id 2d64589ae67cae4a1aef5174e349921a0091f365c9e00d84fa384c08a6b55486
Jul 10 13:26:56.835277 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.835194    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wnk2p" event={"ID":"025769c9-3932-4cd0-97cb-96e766b3157c","Type":"ContainerStarted","Data":"22c388c2c4c620a4b2677010f6ee6f219550cd23080e2c3b00850870658a2f77"}
Jul 10 13:26:56.836390 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.836365    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-22d58" event={"ID":"69f2fb4d-4440-4815-afd9-f1d7186dcafd","Type":"ContainerStarted","Data":"2d64589ae67cae4a1aef5174e349921a0091f365c9e00d84fa384c08a6b55486"}
Jul 10 13:26:56.839060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.839036    2554 generic.go:356] "Generic (PLEG): container finished" podID="186e59bb-668c-4cb7-b099-f0f86ee14eaf" containerID="5dd96f4cbe5e93427397d5a2651a27b57e7c76ac97538dda1409aa391c6e5741" exitCode=0
Jul 10 13:26:56.839142 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.839073    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerDied","Data":"5dd96f4cbe5e93427397d5a2651a27b57e7c76ac97538dda1409aa391c6e5741"}
Jul 10 13:26:56.855508 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:56.855465    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wnk2p" podStartSLOduration=1.477997487 podStartE2EDuration="4.855451406s" totalImagesPullingTime="3.377453919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:56.854861734 +0000 UTC m=+40.720988386" watchObservedRunningTime="2026-07-10 13:26:56.855451406 +0000 UTC m=+40.721578037"
Jul 10 13:26:57.845018 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:57.844983    2554 generic.go:356] "Generic (PLEG): container finished" podID="186e59bb-668c-4cb7-b099-f0f86ee14eaf" containerID="3e47048c8ddbd8e58238cf7799487871a2359145cacddbd5feda0925d6d85701" exitCode=0
Jul 10 13:26:57.845472 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:57.845064    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerDied","Data":"3e47048c8ddbd8e58238cf7799487871a2359145cacddbd5feda0925d6d85701"}
Jul 10 13:26:58.849925 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:58.849891    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" event={"ID":"186e59bb-668c-4cb7-b099-f0f86ee14eaf","Type":"ContainerStarted","Data":"75fc69c9668d364516723bcd9a058a4b8e27769fcde30c9768f99d5d1b803caf"}
Jul 10 13:26:58.880276 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:58.880233    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cmp9c" podStartSLOduration=9.687937134 podStartE2EDuration="42.880219323s" totalImagesPullingTime="33.192282189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:58.880110737 +0000 UTC m=+42.746237390" watchObservedRunningTime="2026-07-10 13:26:58.880219323 +0000 UTC m=+42.746345977"
Jul 10 13:26:59.855311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:59.855271    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-22d58" event={"ID":"69f2fb4d-4440-4815-afd9-f1d7186dcafd","Type":"ContainerStarted","Data":"8179fafc1dedaa7123dd3dad37fc4ed337c00114214d9ce2170dfc10c384bc98"}
Jul 10 13:26:59.876666 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:26:59.876614    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-22d58" podStartSLOduration=4.347538542 podStartE2EDuration="6.876591387s" totalImagesPullingTime="2.529052845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:59.875544758 +0000 UTC m=+43.741671419" watchObservedRunningTime="2026-07-10 13:26:59.876591387 +0000 UTC m=+43.742718042"
Jul 10 13:27:00.323478 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:00.323448    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:00.323663 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:00.323518    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:00.323663 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:00.323556    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:00.323663 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.323600    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 13:27:00.323812 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.323673    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 13:27:00.323812 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.323678    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 13:27:00.323812 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.323693    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5ddb849bbf-2wq92: secret "image-registry-tls" not found
Jul 10 13:27:00.323812 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.323675    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:27:08.32365582 +0000 UTC m=+52.189782467 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : configmap "kube-rbac-proxy" not found
Jul 10 13:27:00.323812 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.323738    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:27:08.323722283 +0000 UTC m=+52.189848931 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : secret "insights-runtime-extractor-tls" not found
Jul 10 13:27:00.323812 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.323750    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls podName:97483560-993f-44e7-8850-341815421e10 nodeName:}" failed. No retries permitted until 2026-07-10 13:27:08.323744057 +0000 UTC m=+52.189870688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls") pod "image-registry-5ddb849bbf-2wq92" (UID: "97483560-993f-44e7-8850-341815421e10") : secret "image-registry-tls" not found
Jul 10 13:27:00.424541 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:00.424510    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:27:00.427110 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.424722    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 13:27:00.427110 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:00.426807    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls podName:f2a7807f-1566-4212-8e13-1e82c45eca50 nodeName:}" failed. No retries permitted until 2026-07-10 13:27:08.426786582 +0000 UTC m=+52.292913239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls") pod "dns-default-brqtm" (UID: "f2a7807f-1566-4212-8e13-1e82c45eca50") : secret "dns-default-metrics-tls" not found
Jul 10 13:27:08.383482 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.383451    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:08.383899 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.383515    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:08.383899 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.383563    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:08.383899 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:08.383666    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 13:27:08.383899 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:08.383744    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls podName:55a58a5f-0542-4a7c-9ebf-403fc515cd0d nodeName:}" failed. No retries permitted until 2026-07-10 13:27:24.383726121 +0000 UTC m=+68.249852770 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjb42" (UID: "55a58a5f-0542-4a7c-9ebf-403fc515cd0d") : secret "insights-runtime-extractor-tls" not found
Jul 10 13:27:08.384121 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.384098    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:08.386265 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.386244    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"image-registry-5ddb849bbf-2wq92\" (UID: \"97483560-993f-44e7-8850-341815421e10\") " pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:08.444014 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.443992    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:08.484023 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.483999    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:27:08.485976 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.485957    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2a7807f-1566-4212-8e13-1e82c45eca50-metrics-tls\") pod \"dns-default-brqtm\" (UID: \"f2a7807f-1566-4212-8e13-1e82c45eca50\") " pod="openshift-dns/dns-default-brqtm"
Jul 10 13:27:08.559205 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.559173    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brqtm"
Jul 10 13:27:08.588746 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.588710    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ddb849bbf-2wq92"]
Jul 10 13:27:08.591037 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:08.591012    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97483560_993f_44e7_8850_341815421e10.slice/crio-596eb639aad1236814672a0b187ad28a85ce3a37776aba15f20374f70dc5c0ca WatchSource:0}: Error finding container 596eb639aad1236814672a0b187ad28a85ce3a37776aba15f20374f70dc5c0ca: Status 404 returned error can't find the container with id 596eb639aad1236814672a0b187ad28a85ce3a37776aba15f20374f70dc5c0ca
Jul 10 13:27:08.702157 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.702104    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brqtm"]
Jul 10 13:27:08.704967 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:08.704935    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a7807f_1566_4212_8e13_1e82c45eca50.slice/crio-6c3888376ac6c537fb14dc7bcfc713fed02075c9bae62c410a8dd83cc85581e8 WatchSource:0}: Error finding container 6c3888376ac6c537fb14dc7bcfc713fed02075c9bae62c410a8dd83cc85581e8: Status 404 returned error can't find the container with id 6c3888376ac6c537fb14dc7bcfc713fed02075c9bae62c410a8dd83cc85581e8
Jul 10 13:27:08.877096 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.877060    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92" event={"ID":"97483560-993f-44e7-8850-341815421e10","Type":"ContainerStarted","Data":"82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef"}
Jul 10 13:27:08.877234 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.877124    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92" event={"ID":"97483560-993f-44e7-8850-341815421e10","Type":"ContainerStarted","Data":"596eb639aad1236814672a0b187ad28a85ce3a37776aba15f20374f70dc5c0ca"}
Jul 10 13:27:08.877234 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.877169    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:08.878393 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.878358    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brqtm" event={"ID":"f2a7807f-1566-4212-8e13-1e82c45eca50","Type":"ContainerStarted","Data":"6c3888376ac6c537fb14dc7bcfc713fed02075c9bae62c410a8dd83cc85581e8"}
Jul 10 13:27:08.898638 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:08.898561    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92" podStartSLOduration=25.898549567 podStartE2EDuration="25.898549567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:08.898028213 +0000 UTC m=+52.764154878" watchObservedRunningTime="2026-07-10 13:27:08.898549567 +0000 UTC m=+52.764676221"
Jul 10 13:27:11.887981 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:11.887951    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brqtm" event={"ID":"f2a7807f-1566-4212-8e13-1e82c45eca50","Type":"ContainerStarted","Data":"12a3a27fea7506cd7a10f3ad34e5e5e5da31093e4ba10fa0ee1841e9437a1999"}
Jul 10 13:27:11.888458 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:11.888000    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brqtm" event={"ID":"f2a7807f-1566-4212-8e13-1e82c45eca50","Type":"ContainerStarted","Data":"b1a63aa75e314df8296517bbd06dfc394971eaf044af0f3e6c7fe76624711ebf"}
Jul 10 13:27:11.888458 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:11.888089    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-brqtm"
Jul 10 13:27:11.909475 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:11.909437    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-brqtm" podStartSLOduration=17.194315364 podStartE2EDuration="19.909425926s" totalImagesPullingTime="2.715110562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:11.908811615 +0000 UTC m=+55.774938260" watchObservedRunningTime="2026-07-10 13:27:11.909425926 +0000 UTC m=+55.775552597"
Jul 10 13:27:14.528384 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.528339    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"]
Jul 10 13:27:14.531448 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.531418    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"]
Jul 10 13:27:14.531587 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.531562    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.533464 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.533442    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-99r2k"]
Jul 10 13:27:14.533585 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.533563    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.534863 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.534809    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 10 13:27:14.534982 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.534954    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 13:27:14.535045 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.535008    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 10 13:27:14.535287 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.535267    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 13:27:14.535361 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.535274    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"]
Jul 10 13:27:14.535471 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.535341    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.537247 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.537218    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-c5c89\""
Jul 10 13:27:14.537383 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.537258    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 10 13:27:14.537482 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.537462    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.538501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.538472    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 13:27:14.538662 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.538498    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4wg67\""
Jul 10 13:27:14.538662 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.538632    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 13:27:14.540154 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.540127    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 10 13:27:14.540245 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.540199    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 10 13:27:14.541204 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.541158    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 10 13:27:14.541413 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.541286    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 10 13:27:14.544441 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.544402    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"]
Jul 10 13:27:14.547090 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.545393    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"]
Jul 10 13:27:14.550094 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.550069    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-99r2k"]
Jul 10 13:27:14.565482 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.565460    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"]
Jul 10 13:27:14.580983 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.580956    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5ddb849bbf-2wq92"]
Jul 10 13:27:14.630556 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630537    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.630649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630568    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e24d0483-f492-46f5-a9e1-40b4638b8b5d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-99r2k\" (UID: \"e24d0483-f492-46f5-a9e1-40b4638b8b5d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.630649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630587    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e24d0483-f492-46f5-a9e1-40b4638b8b5d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-99r2k\" (UID: \"e24d0483-f492-46f5-a9e1-40b4638b8b5d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.630649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630606    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qspz2\" (UniqueName: \"kubernetes.io/projected/35f9035b-a685-4480-a675-e5b8cfbe6232-kube-api-access-qspz2\") pod \"managed-serviceaccount-addon-agent-7c9b75d874-fbl9n\" (UID: \"35f9035b-a685-4480-a675-e5b8cfbe6232\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.630649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630623    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/613955b8-f978-4255-ac39-bfce093c5f6c-tmp\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.630795 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630733    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57kd6\" (UniqueName: \"kubernetes.io/projected/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-kube-api-access-57kd6\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.630850 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630802    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/613955b8-f978-4255-ac39-bfce093c5f6c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.630932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630848    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h26jm\" (UniqueName: \"kubernetes.io/projected/613955b8-f978-4255-ac39-bfce093c5f6c-kube-api-access-h26jm\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.630932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630877    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-ca\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.630932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630901    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.630932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630925    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-hub\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.631100 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630941    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.631100 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.630998    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35f9035b-a685-4480-a675-e5b8cfbe6232-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9b75d874-fbl9n\" (UID: \"35f9035b-a685-4480-a675-e5b8cfbe6232\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.637011 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.636993    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d7b474fb8-bw7wm"]
Jul 10 13:27:14.639007 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.638992    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.658168 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.658150    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d7b474fb8-bw7wm"]
Jul 10 13:27:14.732266 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732242    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.732344 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732273    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35f9035b-a685-4480-a675-e5b8cfbe6232-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9b75d874-fbl9n\" (UID: \"35f9035b-a685-4480-a675-e5b8cfbe6232\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.732344 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732308    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.732344 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732338    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e24d0483-f492-46f5-a9e1-40b4638b8b5d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-99r2k\" (UID: \"e24d0483-f492-46f5-a9e1-40b4638b8b5d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.732485 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732455    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59cc6f56-3d4f-4948-b857-5f18de51b88b-ca-trust-extracted\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.732539 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732496    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e24d0483-f492-46f5-a9e1-40b4638b8b5d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-99r2k\" (UID: \"e24d0483-f492-46f5-a9e1-40b4638b8b5d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.732590 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732551    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qspz2\" (UniqueName: \"kubernetes.io/projected/35f9035b-a685-4480-a675-e5b8cfbe6232-kube-api-access-qspz2\") pod \"managed-serviceaccount-addon-agent-7c9b75d874-fbl9n\" (UID: \"35f9035b-a685-4480-a675-e5b8cfbe6232\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.732640 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732590    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/613955b8-f978-4255-ac39-bfce093c5f6c-tmp\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.732752 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732731    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59cc6f56-3d4f-4948-b857-5f18de51b88b-installation-pull-secrets\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.732844 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732785    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57kd6\" (UniqueName: \"kubernetes.io/projected/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-kube-api-access-57kd6\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.732907 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732843    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-registry-tls\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.732907 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732873    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59cc6f56-3d4f-4948-b857-5f18de51b88b-registry-certificates\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.732907 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732899    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-bound-sa-token\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.733057 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732949    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59cc6f56-3d4f-4948-b857-5f18de51b88b-image-registry-private-configuration\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.733057 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.732976    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59cc6f56-3d4f-4948-b857-5f18de51b88b-trusted-ca\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.733057 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733017    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/613955b8-f978-4255-ac39-bfce093c5f6c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.733057 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733048    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h26jm\" (UniqueName: \"kubernetes.io/projected/613955b8-f978-4255-ac39-bfce093c5f6c-kube-api-access-h26jm\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.733244 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733017    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/613955b8-f978-4255-ac39-bfce093c5f6c-tmp\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.733244 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733205    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-ca\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.733349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733254    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.733349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733309    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kskgz\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-kube-api-access-kskgz\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.733807 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733686    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-hub\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.733992 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.733968    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.734163 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.734143    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e24d0483-f492-46f5-a9e1-40b4638b8b5d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-99r2k\" (UID: \"e24d0483-f492-46f5-a9e1-40b4638b8b5d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.735402 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.735342    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.735499 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.735455    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35f9035b-a685-4480-a675-e5b8cfbe6232-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c9b75d874-fbl9n\" (UID: \"35f9035b-a685-4480-a675-e5b8cfbe6232\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.735690 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.735671    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e24d0483-f492-46f5-a9e1-40b4638b8b5d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-99r2k\" (UID: \"e24d0483-f492-46f5-a9e1-40b4638b8b5d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.735793 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.735680    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/613955b8-f978-4255-ac39-bfce093c5f6c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.736016 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.735991    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.736331 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.736306    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-ca\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.736596 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.736581    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-hub\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.747482 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.747457    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h26jm\" (UniqueName: \"kubernetes.io/projected/613955b8-f978-4255-ac39-bfce093c5f6c-kube-api-access-h26jm\") pod \"klusterlet-addon-workmgr-7cfdb7fb46-btbw5\" (UID: \"613955b8-f978-4255-ac39-bfce093c5f6c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.747578 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.747562    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qspz2\" (UniqueName: \"kubernetes.io/projected/35f9035b-a685-4480-a675-e5b8cfbe6232-kube-api-access-qspz2\") pod \"managed-serviceaccount-addon-agent-7c9b75d874-fbl9n\" (UID: \"35f9035b-a685-4480-a675-e5b8cfbe6232\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.748546 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.748528    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57kd6\" (UniqueName: \"kubernetes.io/projected/6f2fa0ac-074c-4347-a0a4-cbcaf1521f24-kube-api-access-57kd6\") pod \"cluster-proxy-proxy-agent-57f58b5d69-wcr8b\" (UID: \"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.834749 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834698    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59cc6f56-3d4f-4948-b857-5f18de51b88b-ca-trust-extracted\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.834749 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834736    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59cc6f56-3d4f-4948-b857-5f18de51b88b-installation-pull-secrets\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.834924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834759    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-registry-tls\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.834924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834776    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59cc6f56-3d4f-4948-b857-5f18de51b88b-registry-certificates\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.834924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834802    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-bound-sa-token\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.834924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834850    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59cc6f56-3d4f-4948-b857-5f18de51b88b-image-registry-private-configuration\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.834924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834873    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59cc6f56-3d4f-4948-b857-5f18de51b88b-trusted-ca\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.834924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.834912    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kskgz\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-kube-api-access-kskgz\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.835341 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.835298    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/59cc6f56-3d4f-4948-b857-5f18de51b88b-ca-trust-extracted\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.835626 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.835595    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/59cc6f56-3d4f-4948-b857-5f18de51b88b-registry-certificates\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.835974 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.835954    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/59cc6f56-3d4f-4948-b857-5f18de51b88b-trusted-ca\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.836851 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.836807    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-registry-tls\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.837027 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.837012    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/59cc6f56-3d4f-4948-b857-5f18de51b88b-installation-pull-secrets\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.837271 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.837251    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/59cc6f56-3d4f-4948-b857-5f18de51b88b-image-registry-private-configuration\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.845025 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.845006    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:14.850871 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.850854    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-bound-sa-token\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.851042 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.851027    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kskgz\" (UniqueName: \"kubernetes.io/projected/59cc6f56-3d4f-4948-b857-5f18de51b88b-kube-api-access-kskgz\") pod \"image-registry-6d7b474fb8-bw7wm\" (UID: \"59cc6f56-3d4f-4948-b857-5f18de51b88b\") " pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:14.873412 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.873392    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"
Jul 10 13:27:14.878518 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.878488    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k"
Jul 10 13:27:14.883692 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.883672    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"
Jul 10 13:27:14.957439 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:14.957412    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:15.011669 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.010093    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"]
Jul 10 13:27:15.013484 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:15.013453    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613955b8_f978_4255_ac39_bfce093c5f6c.slice/crio-9f82f96202eb767c29f5abdcb9c9b6619779584bc8977679b3a7d657cefe33de WatchSource:0}: Error finding container 9f82f96202eb767c29f5abdcb9c9b6619779584bc8977679b3a7d657cefe33de: Status 404 returned error can't find the container with id 9f82f96202eb767c29f5abdcb9c9b6619779584bc8977679b3a7d657cefe33de
Jul 10 13:27:15.036859 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.036802    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n"]
Jul 10 13:27:15.038502 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:15.038475    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35f9035b_a685_4480_a675_e5b8cfbe6232.slice/crio-77a3c84fcd7e0759c33054e611260d5ab0ed99c1276ebdb5aa485116186b24f4 WatchSource:0}: Error finding container 77a3c84fcd7e0759c33054e611260d5ab0ed99c1276ebdb5aa485116186b24f4: Status 404 returned error can't find the container with id 77a3c84fcd7e0759c33054e611260d5ab0ed99c1276ebdb5aa485116186b24f4
Jul 10 13:27:15.107837 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.107758    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d7b474fb8-bw7wm"]
Jul 10 13:27:15.110478 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:15.110455    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59cc6f56_3d4f_4948_b857_5f18de51b88b.slice/crio-b989d9c7109f95c9da3be48558eb1f2bb4b4ec5245f73461c7a1f43684a03ad1 WatchSource:0}: Error finding container b989d9c7109f95c9da3be48558eb1f2bb4b4ec5245f73461c7a1f43684a03ad1: Status 404 returned error can't find the container with id b989d9c7109f95c9da3be48558eb1f2bb4b4ec5245f73461c7a1f43684a03ad1
Jul 10 13:27:15.259955 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.259932    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b"]
Jul 10 13:27:15.262152 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:15.262117    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f2fa0ac_074c_4347_a0a4_cbcaf1521f24.slice/crio-724b9e83ade9695b13438e148c78415f3815ff3d2ee6634956f35856add49a34 WatchSource:0}: Error finding container 724b9e83ade9695b13438e148c78415f3815ff3d2ee6634956f35856add49a34: Status 404 returned error can't find the container with id 724b9e83ade9695b13438e148c78415f3815ff3d2ee6634956f35856add49a34
Jul 10 13:27:15.266726 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.266699    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-99r2k"]
Jul 10 13:27:15.270232 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:15.270208    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24d0483_f492_46f5_a9e1_40b4638b8b5d.slice/crio-3ccee2c242cd67e176a586f8d97772de266007d3de60a4a6d33a4dddc2f7f141 WatchSource:0}: Error finding container 3ccee2c242cd67e176a586f8d97772de266007d3de60a4a6d33a4dddc2f7f141: Status 404 returned error can't find the container with id 3ccee2c242cd67e176a586f8d97772de266007d3de60a4a6d33a4dddc2f7f141
Jul 10 13:27:15.903839 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.903787    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k" event={"ID":"e24d0483-f492-46f5-a9e1-40b4638b8b5d","Type":"ContainerStarted","Data":"3ccee2c242cd67e176a586f8d97772de266007d3de60a4a6d33a4dddc2f7f141"}
Jul 10 13:27:15.905541 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.905493    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n" event={"ID":"35f9035b-a685-4480-a675-e5b8cfbe6232","Type":"ContainerStarted","Data":"77a3c84fcd7e0759c33054e611260d5ab0ed99c1276ebdb5aa485116186b24f4"}
Jul 10 13:27:15.907896 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.907314    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm" event={"ID":"59cc6f56-3d4f-4948-b857-5f18de51b88b","Type":"ContainerStarted","Data":"31bb43b1011418b4a58663fcb3ec088106b65750d559544bdf068853480b6452"}
Jul 10 13:27:15.907896 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.907366    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm" event={"ID":"59cc6f56-3d4f-4948-b857-5f18de51b88b","Type":"ContainerStarted","Data":"b989d9c7109f95c9da3be48558eb1f2bb4b4ec5245f73461c7a1f43684a03ad1"}
Jul 10 13:27:15.907896 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.907559    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:15.909608 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.909580    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b" event={"ID":"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24","Type":"ContainerStarted","Data":"724b9e83ade9695b13438e148c78415f3815ff3d2ee6634956f35856add49a34"}
Jul 10 13:27:15.911659 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.911636    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5" event={"ID":"613955b8-f978-4255-ac39-bfce093c5f6c","Type":"ContainerStarted","Data":"9f82f96202eb767c29f5abdcb9c9b6619779584bc8977679b3a7d657cefe33de"}
Jul 10 13:27:15.941170 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:15.941053    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm" podStartSLOduration=1.941037823 podStartE2EDuration="1.941037823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:15.939889581 +0000 UTC m=+59.806016236" watchObservedRunningTime="2026-07-10 13:27:15.941037823 +0000 UTC m=+59.807164478"
Jul 10 13:27:17.833796 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:17.832483    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwjcx"
Jul 10 13:27:21.893177 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.893145    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-brqtm"
Jul 10 13:27:21.937441 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.937401    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n" event={"ID":"35f9035b-a685-4480-a675-e5b8cfbe6232","Type":"ContainerStarted","Data":"38706202ccca20a1a0c8a836fa165e2efb48fe884e98973c1f66108e84af0349"}
Jul 10 13:27:21.939781 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.939751    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b" event={"ID":"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24","Type":"ContainerStarted","Data":"82037c932e2a72f06f2cfb8969730e9f12d2fe809ef4114db9e074d255c63fcc"}
Jul 10 13:27:21.939950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.939800    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b" event={"ID":"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24","Type":"ContainerStarted","Data":"f5b6bbcf13b832632a67ec9ebf4243188e8fa4c86ae6746eba80ef19b3476392"}
Jul 10 13:27:21.939950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.939838    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b" event={"ID":"6f2fa0ac-074c-4347-a0a4-cbcaf1521f24","Type":"ContainerStarted","Data":"d14af0eb0cdca361c31bc9d6c07b086e2120f8cb6410b789521993d936fcf5b5"}
Jul 10 13:27:21.941573 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.941550    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5" event={"ID":"613955b8-f978-4255-ac39-bfce093c5f6c","Type":"ContainerStarted","Data":"57c6512bfbb518f788006465faf8652a489118949a3a2a1ad5b1deae6d6a345e"}
Jul 10 13:27:21.941766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.941741    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:21.943364 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.943330    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k" event={"ID":"e24d0483-f492-46f5-a9e1-40b4638b8b5d","Type":"ContainerStarted","Data":"efd8a4ca863664553eba2c8869f9e95439bc1c5c8a4193b7ea2a689daddde423"}
Jul 10 13:27:21.943474 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.943445    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5"
Jul 10 13:27:21.953875 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.953813    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c9b75d874-fbl9n" podStartSLOduration=1.833427616 podStartE2EDuration="7.953797418s" totalImagesPullingTime="6.120369802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:21.953783885 +0000 UTC m=+65.819910539" watchObservedRunningTime="2026-07-10 13:27:21.953797418 +0000 UTC m=+65.819924076"
Jul 10 13:27:21.979935 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.979897    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57f58b5d69-wcr8b" podStartSLOduration=2.042744801 podStartE2EDuration="7.979885202s" totalImagesPullingTime="5.937140401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:21.978528413 +0000 UTC m=+65.844655067" watchObservedRunningTime="2026-07-10 13:27:21.979885202 +0000 UTC m=+65.846011856"
Jul 10 13:27:21.997388 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:21.997354    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7cfdb7fb46-btbw5" podStartSLOduration=1.81247945 podStartE2EDuration="7.997342713s" totalImagesPullingTime="6.184863263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:21.996294456 +0000 UTC m=+65.862421112" watchObservedRunningTime="2026-07-10 13:27:21.997342713 +0000 UTC m=+65.863469367"
Jul 10 13:27:22.015090 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.015052    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-99r2k" podStartSLOduration=2.128042151 podStartE2EDuration="8.015040427s" totalImagesPullingTime="5.886998276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:22.014219707 +0000 UTC m=+65.880346361" watchObservedRunningTime="2026-07-10 13:27:22.015040427 +0000 UTC m=+65.881167080"
Jul 10 13:27:22.400721 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.400689    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:27:22.400894 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.400735    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:27:22.403666 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.403646    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 13:27:22.403760 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.403651    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 13:27:22.413991 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.413968    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce6950c4-e47b-45eb-9ad1-82ae182964b1-metrics-certs\") pod \"network-metrics-daemon-4qt4r\" (UID: \"ce6950c4-e47b-45eb-9ad1-82ae182964b1\") " pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:27:22.414067 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.413968    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0acdcc26-44da-455e-a2df-a871c3c9a79f-original-pull-secret\") pod \"global-pull-secret-syncer-ggn9l\" (UID: \"0acdcc26-44da-455e-a2df-a871c3c9a79f\") " pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:27:22.502004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.501972    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:27:22.504997 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.504982    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 13:27:22.514434 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.514420    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 13:27:22.524459 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.524431    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcsbw\" (UniqueName: \"kubernetes.io/projected/f4047511-c962-4549-a800-86acd8d84803-kube-api-access-bcsbw\") pod \"network-check-target-8swsb\" (UID: \"f4047511-c962-4549-a800-86acd8d84803\") " pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:27:22.668763 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.668715    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ggn9l"
Jul 10 13:27:22.678356 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.678334    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4gcrn\""
Jul 10 13:27:22.683884 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.683864    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lkjb8\""
Jul 10 13:27:22.686769 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.686751    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4qt4r"
Jul 10 13:27:22.692596 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.692578    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:27:22.810233 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.810202    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ggn9l"]
Jul 10 13:27:22.813362 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:22.813332    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acdcc26_44da_455e_a2df_a871c3c9a79f.slice/crio-49c49dd65350960cea68b1c83ccea8e13be3407faf778a2382e65e2d492bde5c WatchSource:0}: Error finding container 49c49dd65350960cea68b1c83ccea8e13be3407faf778a2382e65e2d492bde5c: Status 404 returned error can't find the container with id 49c49dd65350960cea68b1c83ccea8e13be3407faf778a2382e65e2d492bde5c
Jul 10 13:27:22.827681 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.827646    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4qt4r"]
Jul 10 13:27:22.831038 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:22.831013    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6950c4_e47b_45eb_9ad1_82ae182964b1.slice/crio-e6f61f75df0d59db5f5fd2bbe3b6c1a4e2ea84607af38876497638064265a742 WatchSource:0}: Error finding container e6f61f75df0d59db5f5fd2bbe3b6c1a4e2ea84607af38876497638064265a742: Status 404 returned error can't find the container with id e6f61f75df0d59db5f5fd2bbe3b6c1a4e2ea84607af38876497638064265a742
Jul 10 13:27:22.844427 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.844406    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8swsb"]
Jul 10 13:27:22.847778 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:22.847758    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4047511_c962_4549_a800_86acd8d84803.slice/crio-292639f09048f6dfe8c713ecc46a78d255d4858453e4c6b56ec63a9f9c7b9c9c WatchSource:0}: Error finding container 292639f09048f6dfe8c713ecc46a78d255d4858453e4c6b56ec63a9f9c7b9c9c: Status 404 returned error can't find the container with id 292639f09048f6dfe8c713ecc46a78d255d4858453e4c6b56ec63a9f9c7b9c9c
Jul 10 13:27:22.947421 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.947395    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4qt4r" event={"ID":"ce6950c4-e47b-45eb-9ad1-82ae182964b1","Type":"ContainerStarted","Data":"e6f61f75df0d59db5f5fd2bbe3b6c1a4e2ea84607af38876497638064265a742"}
Jul 10 13:27:22.948532 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.948515    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8swsb" event={"ID":"f4047511-c962-4549-a800-86acd8d84803","Type":"ContainerStarted","Data":"292639f09048f6dfe8c713ecc46a78d255d4858453e4c6b56ec63a9f9c7b9c9c"}
Jul 10 13:27:22.949487 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:22.949451    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ggn9l" event={"ID":"0acdcc26-44da-455e-a2df-a871c3c9a79f","Type":"ContainerStarted","Data":"49c49dd65350960cea68b1c83ccea8e13be3407faf778a2382e65e2d492bde5c"}
Jul 10 13:27:24.417626 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.417526    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:24.420739 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.420713    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/55a58a5f-0542-4a7c-9ebf-403fc515cd0d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjb42\" (UID: \"55a58a5f-0542-4a7c-9ebf-403fc515cd0d\") " pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:24.588041 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.587988    2554 patch_prober.go:28] interesting pod/image-registry-5ddb849bbf-2wq92 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 13:27:24.588213 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.588074    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92" podUID="97483560-993f-44e7-8850-341815421e10" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 13:27:24.661782 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.661751    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wjzj9\""
Jul 10 13:27:24.669760 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.669684    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hjb42"
Jul 10 13:27:24.848702 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.848637    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hjb42"]
Jul 10 13:27:24.851985 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:24.851946    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55a58a5f_0542_4a7c_9ebf_403fc515cd0d.slice/crio-d95ba84647d839faa26770356f006a47b45301f0a2082a62931279d417517a00 WatchSource:0}: Error finding container d95ba84647d839faa26770356f006a47b45301f0a2082a62931279d417517a00: Status 404 returned error can't find the container with id d95ba84647d839faa26770356f006a47b45301f0a2082a62931279d417517a00
Jul 10 13:27:24.962185 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.962084    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4qt4r" event={"ID":"ce6950c4-e47b-45eb-9ad1-82ae182964b1","Type":"ContainerStarted","Data":"0dcd539eed52d54679c620e849a071d4e331b4d7cac44399315ab502a057d303"}
Jul 10 13:27:24.962185 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.962147    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4qt4r" event={"ID":"ce6950c4-e47b-45eb-9ad1-82ae182964b1","Type":"ContainerStarted","Data":"e818993b4c5e7a878b3c0a87e5ecae3ab19a2cd72504fbbb382b913b31363850"}
Jul 10 13:27:24.964483 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.964454    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjb42" event={"ID":"55a58a5f-0542-4a7c-9ebf-403fc515cd0d","Type":"ContainerStarted","Data":"d95ba84647d839faa26770356f006a47b45301f0a2082a62931279d417517a00"}
Jul 10 13:27:24.980405 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:24.980356    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4qt4r" podStartSLOduration=67.74028808 podStartE2EDuration="1m8.980339345s" totalImagesPullingTime="1.240051265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:24.979603766 +0000 UTC m=+68.845730421" watchObservedRunningTime="2026-07-10 13:27:24.980339345 +0000 UTC m=+68.846466000"
Jul 10 13:27:30.986835 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:30.986787    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjb42" event={"ID":"55a58a5f-0542-4a7c-9ebf-403fc515cd0d","Type":"ContainerStarted","Data":"b169ee5fa670de3b4966285a1182e1d451c7b4a036953efe768c4a44c4649e6f"}
Jul 10 13:27:30.988303 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:30.988273    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8swsb" event={"ID":"f4047511-c962-4549-a800-86acd8d84803","Type":"ContainerStarted","Data":"d88b89aed5ad4f52e043fd4a3f093f137ea7c457ebe9bf77d5ed3a2b24361454"}
Jul 10 13:27:30.988435 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:30.988395    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:27:30.989862 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:30.989809    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ggn9l" event={"ID":"0acdcc26-44da-455e-a2df-a871c3c9a79f","Type":"ContainerStarted","Data":"ab0d57cef32184b9e4221223c980fcfd3ed68d650fecc65cc200ff27d6c1b6df"}
Jul 10 13:27:31.006272 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:31.006229    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8swsb" podStartSLOduration=67.214672444 podStartE2EDuration="1m15.006216263s" totalImagesPullingTime="7.791543819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:31.005492019 +0000 UTC m=+74.871618672" watchObservedRunningTime="2026-07-10 13:27:31.006216263 +0000 UTC m=+74.872342917"
Jul 10 13:27:31.023314 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:31.023269    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ggn9l" podStartSLOduration=67.197386612 podStartE2EDuration="1m15.023257473s" totalImagesPullingTime="7.825870861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:31.022269139 +0000 UTC m=+74.888395794" watchObservedRunningTime="2026-07-10 13:27:31.023257473 +0000 UTC m=+74.889384126"
Jul 10 13:27:31.994785 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:31.994754    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjb42" event={"ID":"55a58a5f-0542-4a7c-9ebf-403fc515cd0d","Type":"ContainerStarted","Data":"88ac33acf9f09bf869fbc4463890d979df807c2da756f840336db6b10af14efa"}
Jul 10 13:27:32.233440 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.233338    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h4hgg"]
Jul 10 13:27:32.235900 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.235878    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.238629 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.238604    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 13:27:32.238766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.238748    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 13:27:32.238849 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.238611    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 13:27:32.239224 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.239064    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7qstg\""
Jul 10 13:27:32.240053 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.240028    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 13:27:32.240159 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.240141    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 13:27:32.240406 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.240164    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 13:27:32.385214 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385146    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-textfile\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385214 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385182    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-root\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385214 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385198    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54g2l\" (UniqueName: \"kubernetes.io/projected/9447a7eb-4121-439a-95a8-6fbe7f162689-kube-api-access-54g2l\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385274    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385314    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9447a7eb-4121-439a-95a8-6fbe7f162689-metrics-client-ca\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385344    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-sys\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385376    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-tls\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385424    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-wtmp\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.385463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.385451    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486148 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486121    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-root\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486161    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54g2l\" (UniqueName: \"kubernetes.io/projected/9447a7eb-4121-439a-95a8-6fbe7f162689-kube-api-access-54g2l\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486183    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486205    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9447a7eb-4121-439a-95a8-6fbe7f162689-metrics-client-ca\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486230    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-sys\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486236    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-root\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486258    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-tls\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486284    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-wtmp\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486639 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486309    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486639 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486368    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-textfile\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486742 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486653    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-sys\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486742 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486685    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-wtmp\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486871 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486838    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-accelerators-collector-config\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486979 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486845    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-textfile\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.486979 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.486955    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9447a7eb-4121-439a-95a8-6fbe7f162689-metrics-client-ca\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.488971 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.488944    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-tls\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.489148 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.489130    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9447a7eb-4121-439a-95a8-6fbe7f162689-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.500477 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.500449    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54g2l\" (UniqueName: \"kubernetes.io/projected/9447a7eb-4121-439a-95a8-6fbe7f162689-kube-api-access-54g2l\") pod \"node-exporter-h4hgg\" (UID: \"9447a7eb-4121-439a-95a8-6fbe7f162689\") " pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.547554 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:32.547532    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h4hgg"
Jul 10 13:27:32.559446 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:27:32.559418    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9447a7eb_4121_439a_95a8_6fbe7f162689.slice/crio-924549935c7f3fe8e8503796c516acde763218ceebbbc44456b938de9c182839 WatchSource:0}: Error finding container 924549935c7f3fe8e8503796c516acde763218ceebbbc44456b938de9c182839: Status 404 returned error can't find the container with id 924549935c7f3fe8e8503796c516acde763218ceebbbc44456b938de9c182839
Jul 10 13:27:33.001627 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:33.001594    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4hgg" event={"ID":"9447a7eb-4121-439a-95a8-6fbe7f162689","Type":"ContainerStarted","Data":"924549935c7f3fe8e8503796c516acde763218ceebbbc44456b938de9c182839"}
Jul 10 13:27:34.585139 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:34.585121    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:35.010421 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:35.010386    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjb42" event={"ID":"55a58a5f-0542-4a7c-9ebf-403fc515cd0d","Type":"ContainerStarted","Data":"49b31d38122ff6fc8f39f1a5771fa2461a0b762b6449c79f3bcca46cbce39744"}
Jul 10 13:27:35.012047 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:35.012023    2554 generic.go:356] "Generic (PLEG): container finished" podID="9447a7eb-4121-439a-95a8-6fbe7f162689" containerID="3c3658f63227cf9dc6cbdf123ae6330fc40782d7cffd9207c810da9f8a2cf428" exitCode=0
Jul 10 13:27:35.012153 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:35.012085    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4hgg" event={"ID":"9447a7eb-4121-439a-95a8-6fbe7f162689","Type":"ContainerDied","Data":"3c3658f63227cf9dc6cbdf123ae6330fc40782d7cffd9207c810da9f8a2cf428"}
Jul 10 13:27:35.031509 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:35.031465    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hjb42" podStartSLOduration=33.486919903 podStartE2EDuration="43.031452196s" totalImagesPullingTime="9.544532293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:26:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:35.03073324 +0000 UTC m=+78.896859896" watchObservedRunningTime="2026-07-10 13:27:35.031452196 +0000 UTC m=+78.897578849"
Jul 10 13:27:36.018090 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:36.018046    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4hgg" event={"ID":"9447a7eb-4121-439a-95a8-6fbe7f162689","Type":"ContainerStarted","Data":"bfa20f245821c62d9ce7a71bed7b24fc4aa35f4354938e014a14484b3ed7950f"}
Jul 10 13:27:36.018550 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:36.018136    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h4hgg" event={"ID":"9447a7eb-4121-439a-95a8-6fbe7f162689","Type":"ContainerStarted","Data":"d6c61511f2b2ab1315a850e4679052f872f92a5d5b9c9115aebd0bfb2bbf5d4d"}
Jul 10 13:27:36.041506 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:36.041459    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h4hgg" podStartSLOduration=2.026222811 podStartE2EDuration="4.041442542s" totalImagesPullingTime="1.949476789s" totalInitContainerRuntime="65.742942ms" isStatefulPod=true podCreationTimestamp="2026-07-10 13:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:36.040059196 +0000 UTC m=+79.906185851" watchObservedRunningTime="2026-07-10 13:27:36.041442542 +0000 UTC m=+79.907569196"
Jul 10 13:27:36.920790 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:36.920758    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d7b474fb8-bw7wm"
Jul 10 13:27:39.600645 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.600525    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92" podUID="97483560-993f-44e7-8850-341815421e10" containerName="registry" containerID="cri-o://82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef" gracePeriod=30
Jul 10 13:27:39.838315 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.838293    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:39.940993 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.940971    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m5z9\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-kube-api-access-8m5z9\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941119 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941019    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-trusted-ca\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941119 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941039    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-registry-certificates\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941119 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941069    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941119 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941104    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-image-registry-private-configuration\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941320 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941140    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-installation-pull-secrets\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941320 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941171    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-bound-sa-token\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941320 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941227    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97483560-993f-44e7-8850-341815421e10-ca-trust-extracted\") pod \"97483560-993f-44e7-8850-341815421e10\" (UID: \"97483560-993f-44e7-8850-341815421e10\") "
Jul 10 13:27:39.941437 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941408    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:27:39.941484 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.941465    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:27:39.943793 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.943751    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:27:39.943793 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.943765    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:27:39.943981 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.943840    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-kube-api-access-8m5z9" (OuterVolumeSpecName: "kube-api-access-8m5z9") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "kube-api-access-8m5z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:27:39.944076 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.944058    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:27:39.944128 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.944095    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:27:39.949791 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:39.949770    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97483560-993f-44e7-8850-341815421e10-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "97483560-993f-44e7-8850-341815421e10" (UID: "97483560-993f-44e7-8850-341815421e10"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:27:40.033362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.033334    2554 generic.go:356] "Generic (PLEG): container finished" podID="97483560-993f-44e7-8850-341815421e10" containerID="82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef" exitCode=0
Jul 10 13:27:40.033455 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.033375    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92"
Jul 10 13:27:40.033455 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.033407    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92" event={"ID":"97483560-993f-44e7-8850-341815421e10","Type":"ContainerDied","Data":"82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef"}
Jul 10 13:27:40.033455 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.033448    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ddb849bbf-2wq92" event={"ID":"97483560-993f-44e7-8850-341815421e10","Type":"ContainerDied","Data":"596eb639aad1236814672a0b187ad28a85ce3a37776aba15f20374f70dc5c0ca"}
Jul 10 13:27:40.033566 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.033468    2554 scope.go:117] "RemoveContainer" containerID="82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef"
Jul 10 13:27:40.041863 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041847    2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97483560-993f-44e7-8850-341815421e10-ca-trust-extracted\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.041922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041867    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m5z9\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-kube-api-access-8m5z9\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.041922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041879    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-trusted-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.041922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041887    2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97483560-993f-44e7-8850-341815421e10-registry-certificates\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.041922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041895    2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-registry-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.041922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041904    2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-image-registry-private-configuration\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.041922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041913    2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97483560-993f-44e7-8850-341815421e10-installation-pull-secrets\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.041922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041921    2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97483560-993f-44e7-8850-341815421e10-bound-sa-token\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:27:40.042161 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.041893    2554 scope.go:117] "RemoveContainer" containerID="82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef"
Jul 10 13:27:40.042198 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:27:40.042176    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef\": container with ID starting with 82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef not found: ID does not exist" containerID="82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef"
Jul 10 13:27:40.042236 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.042206    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef"} err="failed to get container status \"82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef\": rpc error: code = NotFound desc = could not find container \"82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef\": container with ID starting with 82c7a9ec19fd5740c7c008574ee418c1e8c790d57e53165d3142187456f1adef not found: ID does not exist"
Jul 10 13:27:40.055531 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.055504    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5ddb849bbf-2wq92"]
Jul 10 13:27:40.063224 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.063205    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5ddb849bbf-2wq92"]
Jul 10 13:27:40.663728 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:27:40.663694    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97483560-993f-44e7-8850-341815421e10" path="/var/lib/kubelet/pods/97483560-993f-44e7-8850-341815421e10/volumes"
Jul 10 13:28:01.998081 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:28:01.998054    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8swsb"
Jul 10 13:31:16.502533 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:31:16.502501    2554 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 13:32:38.550621 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.550545    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"]
Jul 10 13:32:38.551070 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.550786    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97483560-993f-44e7-8850-341815421e10" containerName="registry"
Jul 10 13:32:38.551070 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.550796    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="97483560-993f-44e7-8850-341815421e10" containerName="registry"
Jul 10 13:32:38.551070 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.550851    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="97483560-993f-44e7-8850-341815421e10" containerName="registry"
Jul 10 13:32:38.552623 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.552604    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.556996 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.556973    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 13:32:38.557734 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.557718    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-66n66\""
Jul 10 13:32:38.558204 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.558183    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 13:32:38.558292 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.558228    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 13:32:38.560936 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.560918    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 13:32:38.574871 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.574849    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"]
Jul 10 13:32:38.643991 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.643973    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac206005-8ec2-4863-b3a2-8f28b36f32a1-manifests\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.644083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.643998    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac206005-8ec2-4863-b3a2-8f28b36f32a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.644083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.644027    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac206005-8ec2-4863-b3a2-8f28b36f32a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.644083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.644061    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac206005-8ec2-4863-b3a2-8f28b36f32a1-tmp\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.644185 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.644084    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zhsz\" (UniqueName: \"kubernetes.io/projected/ac206005-8ec2-4863-b3a2-8f28b36f32a1-kube-api-access-4zhsz\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.744711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.744688    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac206005-8ec2-4863-b3a2-8f28b36f32a1-manifests\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.744809 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.744714    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac206005-8ec2-4863-b3a2-8f28b36f32a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.744809 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.744736    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac206005-8ec2-4863-b3a2-8f28b36f32a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.744809 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.744753    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac206005-8ec2-4863-b3a2-8f28b36f32a1-tmp\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.744809 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.744783    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zhsz\" (UniqueName: \"kubernetes.io/projected/ac206005-8ec2-4863-b3a2-8f28b36f32a1-kube-api-access-4zhsz\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.745089 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.745071    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac206005-8ec2-4863-b3a2-8f28b36f32a1-manifests\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.747013 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.746982    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac206005-8ec2-4863-b3a2-8f28b36f32a1-tmp\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.747139 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.747122    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac206005-8ec2-4863-b3a2-8f28b36f32a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.747198 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.747175    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac206005-8ec2-4863-b3a2-8f28b36f32a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.757230 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.757207    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zhsz\" (UniqueName: \"kubernetes.io/projected/ac206005-8ec2-4863-b3a2-8f28b36f32a1-kube-api-access-4zhsz\") pod \"opendatahub-operator-controller-manager-7d67ccc8fd-mf285\" (UID: \"ac206005-8ec2-4863-b3a2-8f28b36f32a1\") " pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:38.862928 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:38.862878    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:39.013517 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:39.013495    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"]
Jul 10 13:32:39.016136 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:32:39.016101    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac206005_8ec2_4863_b3a2_8f28b36f32a1.slice/crio-5030a6e25a0c9487b28d583ea70b63b32a8ac1e9723e1a59f2254e640afa2c61 WatchSource:0}: Error finding container 5030a6e25a0c9487b28d583ea70b63b32a8ac1e9723e1a59f2254e640afa2c61: Status 404 returned error can't find the container with id 5030a6e25a0c9487b28d583ea70b63b32a8ac1e9723e1a59f2254e640afa2c61
Jul 10 13:32:39.017930 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:39.017914    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 13:32:39.049584 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:39.049560    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285" event={"ID":"ac206005-8ec2-4863-b3a2-8f28b36f32a1","Type":"ContainerStarted","Data":"5030a6e25a0c9487b28d583ea70b63b32a8ac1e9723e1a59f2254e640afa2c61"}
Jul 10 13:32:42.061641 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:42.061608    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285" event={"ID":"ac206005-8ec2-4863-b3a2-8f28b36f32a1","Type":"ContainerStarted","Data":"53d00e127cf425ca32becdc8e9bb60618873ecc02025cbfaa9aaad5913722c28"}
Jul 10 13:32:43.066885 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:43.066851    2554 generic.go:356] "Generic (PLEG): container finished" podID="ac206005-8ec2-4863-b3a2-8f28b36f32a1" containerID="53d00e127cf425ca32becdc8e9bb60618873ecc02025cbfaa9aaad5913722c28" exitCode=0
Jul 10 13:32:43.067224 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:43.066913    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285" event={"ID":"ac206005-8ec2-4863-b3a2-8f28b36f32a1","Type":"ContainerDied","Data":"53d00e127cf425ca32becdc8e9bb60618873ecc02025cbfaa9aaad5913722c28"}
Jul 10 13:32:44.072497 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:44.072459    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285" event={"ID":"ac206005-8ec2-4863-b3a2-8f28b36f32a1","Type":"ContainerStarted","Data":"556eceb97af5f2318853b94e04530d37049d028e09ac60b8bc82c64d989e7987"}
Jul 10 13:32:44.073113 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:44.072562    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:44.097066 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:44.096999    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285" podStartSLOduration=1.8046080469999999 podStartE2EDuration="6.096980186s" totalImagesPullingTime="3.214156109s" totalInitContainerRuntime="1.07821603s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:32:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:32:44.095110854 +0000 UTC m=+387.961237506" watchObservedRunningTime="2026-07-10 13:32:44.096980186 +0000 UTC m=+387.963106840"
Jul 10 13:32:55.078182 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:55.078149    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d67ccc8fd-mf285"
Jul 10 13:32:57.916496 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:57.916465    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"]
Jul 10 13:32:57.925904 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:57.925884    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:57.929116 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:57.929096    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 13:32:57.929235 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:57.929098    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 13:32:57.930217 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:57.930199    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 13:32:57.930309 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:57.930232    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 13:32:57.941260 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:57.941239    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"]
Jul 10 13:32:58.080273 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.080244    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/690dfb48-6746-450d-8684-9fde68fc75ab-tls-certs\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.080429 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.080289    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/690dfb48-6746-450d-8684-9fde68fc75ab-tmp\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.080429 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.080338    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpn89\" (UniqueName: \"kubernetes.io/projected/690dfb48-6746-450d-8684-9fde68fc75ab-kube-api-access-kpn89\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.181128 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.181105    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/690dfb48-6746-450d-8684-9fde68fc75ab-tmp\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.181249 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.181146    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpn89\" (UniqueName: \"kubernetes.io/projected/690dfb48-6746-450d-8684-9fde68fc75ab-kube-api-access-kpn89\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.181249 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.181182    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/690dfb48-6746-450d-8684-9fde68fc75ab-tls-certs\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.183312 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.183281    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/690dfb48-6746-450d-8684-9fde68fc75ab-tmp\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.183465 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.183449    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/690dfb48-6746-450d-8684-9fde68fc75ab-tls-certs\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.199765 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.199741    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpn89\" (UniqueName: \"kubernetes.io/projected/690dfb48-6746-450d-8684-9fde68fc75ab-kube-api-access-kpn89\") pod \"kube-auth-proxy-6c4978d7b8-6jwst\" (UID: \"690dfb48-6746-450d-8684-9fde68fc75ab\") " pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.203796 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.203770    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-24j8m"]
Jul 10 13:32:58.206218 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.206197    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:58.209639 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.209623    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 10 13:32:58.209711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.209692    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-fr588\""
Jul 10 13:32:58.225968 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.225929    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-24j8m"]
Jul 10 13:32:58.234769 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.234746    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"
Jul 10 13:32:58.282198 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.282162    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgdxj\" (UniqueName: \"kubernetes.io/projected/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-kube-api-access-cgdxj\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:58.282355 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.282220    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:58.383469 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.383435    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgdxj\" (UniqueName: \"kubernetes.io/projected/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-kube-api-access-cgdxj\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:58.383665 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.383481    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:58.383665 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:32:58.383588    2554 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 10 13:32:58.383665 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:32:58.383655    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert podName:f6d92133-a0b4-4d37-ab3f-f2ef318d4664 nodeName:}" failed. No retries permitted until 2026-07-10 13:32:58.88363871 +0000 UTC m=+402.749765360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert") pod "odh-model-controller-7998c67dc4-24j8m" (UID: "f6d92133-a0b4-4d37-ab3f-f2ef318d4664") : secret "odh-model-controller-webhook-cert" not found
Jul 10 13:32:58.400152 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.400121    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgdxj\" (UniqueName: \"kubernetes.io/projected/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-kube-api-access-cgdxj\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:58.416969 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.416944    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst"]
Jul 10 13:32:58.420116 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:32:58.420088    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690dfb48_6746_450d_8684_9fde68fc75ab.slice/crio-b11d51dddcc183164355ac2e999301b6864d91ff5d8e5cedca322f0bb7f8a159 WatchSource:0}: Error finding container b11d51dddcc183164355ac2e999301b6864d91ff5d8e5cedca322f0bb7f8a159: Status 404 returned error can't find the container with id b11d51dddcc183164355ac2e999301b6864d91ff5d8e5cedca322f0bb7f8a159
Jul 10 13:32:58.887766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:58.887728    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:58.887985 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:32:58.887897    2554 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 10 13:32:58.887985 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:32:58.887961    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert podName:f6d92133-a0b4-4d37-ab3f-f2ef318d4664 nodeName:}" failed. No retries permitted until 2026-07-10 13:32:59.887945201 +0000 UTC m=+403.754071834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert") pod "odh-model-controller-7998c67dc4-24j8m" (UID: "f6d92133-a0b4-4d37-ab3f-f2ef318d4664") : secret "odh-model-controller-webhook-cert" not found
Jul 10 13:32:59.125550 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.125510    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9cb55"]
Jul 10 13:32:59.128554 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.128530    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst" event={"ID":"690dfb48-6746-450d-8684-9fde68fc75ab","Type":"ContainerStarted","Data":"b11d51dddcc183164355ac2e999301b6864d91ff5d8e5cedca322f0bb7f8a159"}
Jul 10 13:32:59.128677 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.128612    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.132118 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.132095    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-s6cbc\""
Jul 10 13:32:59.132279 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.132263    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 10 13:32:59.145026 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.144972    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9cb55"]
Jul 10 13:32:59.189809 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.189788    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szg4j\" (UniqueName: \"kubernetes.io/projected/32cbb613-77d9-4690-8c05-9f7a3f78ddfc-kube-api-access-szg4j\") pod \"model-serving-api-54d89bd5fc-9cb55\" (UID: \"32cbb613-77d9-4690-8c05-9f7a3f78ddfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.189930 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.189890    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32cbb613-77d9-4690-8c05-9f7a3f78ddfc-tls-certs\") pod \"model-serving-api-54d89bd5fc-9cb55\" (UID: \"32cbb613-77d9-4690-8c05-9f7a3f78ddfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.290983 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.290958    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32cbb613-77d9-4690-8c05-9f7a3f78ddfc-tls-certs\") pod \"model-serving-api-54d89bd5fc-9cb55\" (UID: \"32cbb613-77d9-4690-8c05-9f7a3f78ddfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.291146 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.291120    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szg4j\" (UniqueName: \"kubernetes.io/projected/32cbb613-77d9-4690-8c05-9f7a3f78ddfc-kube-api-access-szg4j\") pod \"model-serving-api-54d89bd5fc-9cb55\" (UID: \"32cbb613-77d9-4690-8c05-9f7a3f78ddfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.293728 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.293637    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32cbb613-77d9-4690-8c05-9f7a3f78ddfc-tls-certs\") pod \"model-serving-api-54d89bd5fc-9cb55\" (UID: \"32cbb613-77d9-4690-8c05-9f7a3f78ddfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.303652 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.303626    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szg4j\" (UniqueName: \"kubernetes.io/projected/32cbb613-77d9-4690-8c05-9f7a3f78ddfc-kube-api-access-szg4j\") pod \"model-serving-api-54d89bd5fc-9cb55\" (UID: \"32cbb613-77d9-4690-8c05-9f7a3f78ddfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.439577 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.439551    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:32:59.517004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.516972    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-65h44"]
Jul 10 13:32:59.519093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.519071    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.521909 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.521891    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 10 13:32:59.522016 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.521891    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 10 13:32:59.522266 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.522239    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 10 13:32:59.522349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.522275    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 13:32:59.522398 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.522370    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gp66z\""
Jul 10 13:32:59.535968 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.535945    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-65h44"]
Jul 10 13:32:59.582727 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.582701    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9cb55"]
Jul 10 13:32:59.585897 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:32:59.585866    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32cbb613_77d9_4690_8c05_9f7a3f78ddfc.slice/crio-7dd372f522179362b7b046878bfaf256de23960af5ae014c29b620a13715ce8f WatchSource:0}: Error finding container 7dd372f522179362b7b046878bfaf256de23960af5ae014c29b620a13715ce8f: Status 404 returned error can't find the container with id 7dd372f522179362b7b046878bfaf256de23960af5ae014c29b620a13715ce8f
Jul 10 13:32:59.593546 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.593521    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcmxb\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-kube-api-access-gcmxb\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.593652 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.593616    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-local-certs\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.593703 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.593673    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.593751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.593700    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-cacerts\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.593751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.593738    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.593873 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.593842    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.593925 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.593888    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-token\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695121 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695035    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695121 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695085    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-token\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695123    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcmxb\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-kube-api-access-gcmxb\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695186    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-local-certs\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695216    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695242    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-cacerts\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695276    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.695835 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.695782    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.698297 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.698277    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-local-certs\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.698403 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.698274    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.698403 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.698329    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.698403 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.698327    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-cacerts\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.704559 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.704529    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcmxb\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-kube-api-access-gcmxb\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.705075 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.705056    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-token\") pod \"istiod-openshift-gateway-7574459656-65h44\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.829372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.829314    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:32:59.898978 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.898933    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:59.901479 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.901429    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6d92133-a0b4-4d37-ab3f-f2ef318d4664-cert\") pod \"odh-model-controller-7998c67dc4-24j8m\" (UID: \"f6d92133-a0b4-4d37-ab3f-f2ef318d4664\") " pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:32:59.970604 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:32:59.970532    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-65h44"]
Jul 10 13:32:59.974308 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:32:59.974279    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926ddb3e_0a81_4b7b_be7b_9a1a1b31094a.slice/crio-d790f7ccc7f2ffe920e70433a2b2553db093bf4982fd0c3282341c925c7dd727 WatchSource:0}: Error finding container d790f7ccc7f2ffe920e70433a2b2553db093bf4982fd0c3282341c925c7dd727: Status 404 returned error can't find the container with id d790f7ccc7f2ffe920e70433a2b2553db093bf4982fd0c3282341c925c7dd727
Jul 10 13:33:00.016189 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:00.016166    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:33:00.132680 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:00.132635    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" event={"ID":"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a","Type":"ContainerStarted","Data":"d790f7ccc7f2ffe920e70433a2b2553db093bf4982fd0c3282341c925c7dd727"}
Jul 10 13:33:00.134002 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:00.133970    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9cb55" event={"ID":"32cbb613-77d9-4690-8c05-9f7a3f78ddfc","Type":"ContainerStarted","Data":"7dd372f522179362b7b046878bfaf256de23960af5ae014c29b620a13715ce8f"}
Jul 10 13:33:00.163250 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:00.163224    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-24j8m"]
Jul 10 13:33:00.166446 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:33:00.166411    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d92133_a0b4_4d37_ab3f_f2ef318d4664.slice/crio-ef97df80b3bd74a5a60a42ab07e4ffb08fe39e26366c56c3daebab287a810721 WatchSource:0}: Error finding container ef97df80b3bd74a5a60a42ab07e4ffb08fe39e26366c56c3daebab287a810721: Status 404 returned error can't find the container with id ef97df80b3bd74a5a60a42ab07e4ffb08fe39e26366c56c3daebab287a810721
Jul 10 13:33:01.144673 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:01.144594    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-24j8m" event={"ID":"f6d92133-a0b4-4d37-ab3f-f2ef318d4664","Type":"ContainerStarted","Data":"ef97df80b3bd74a5a60a42ab07e4ffb08fe39e26366c56c3daebab287a810721"}
Jul 10 13:33:04.842473 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:04.842432    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 13:33:04.842852 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:04.842507    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 13:33:05.164521 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.164438    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst" event={"ID":"690dfb48-6746-450d-8684-9fde68fc75ab","Type":"ContainerStarted","Data":"930cef4e9baec1816cf3c0d471b2fdb4a24d1650a67416b216e3764ae02225b3"}
Jul 10 13:33:05.166149 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.166120    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" event={"ID":"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a","Type":"ContainerStarted","Data":"e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631"}
Jul 10 13:33:05.166318 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.166298    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:33:05.167960 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.167896    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9cb55" event={"ID":"32cbb613-77d9-4690-8c05-9f7a3f78ddfc","Type":"ContainerStarted","Data":"3ab36e62e8450bf79ab6d785f564146559d0b676fe63fce4bb15a64a46f45ba5"}
Jul 10 13:33:05.167960 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.167929    2554 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-65h44 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 13:33:05.167960 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.167956    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:33:05.168193 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.167969    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" podUID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 13:33:05.169938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.169919    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-24j8m" event={"ID":"f6d92133-a0b4-4d37-ab3f-f2ef318d4664","Type":"ContainerStarted","Data":"37e4ae2f632c9ebf2aeb69030c1c743257c157c781b0400e53ecfed89dac6bf2"}
Jul 10 13:33:05.170051 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.170034    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:33:05.201167 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.201100    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c4978d7b8-6jwst" podStartSLOduration=1.786932528 podStartE2EDuration="8.201081474s" totalImagesPullingTime="6.414148946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:33:05.182161016 +0000 UTC m=+409.048287669" watchObservedRunningTime="2026-07-10 13:33:05.201081474 +0000 UTC m=+409.067208128"
Jul 10 13:33:05.201343 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.201214    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-24j8m" podStartSLOduration=2.51236925 podStartE2EDuration="7.201207104s" totalImagesPullingTime="4.688837854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:33:05.199004539 +0000 UTC m=+409.065131196" watchObservedRunningTime="2026-07-10 13:33:05.201207104 +0000 UTC m=+409.067333759"
Jul 10 13:33:05.224892 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.224840    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" podStartSLOduration=1.3596786330000001 podStartE2EDuration="6.224800704s" totalImagesPullingTime="4.865122071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:33:05.223488924 +0000 UTC m=+409.089615579" watchObservedRunningTime="2026-07-10 13:33:05.224800704 +0000 UTC m=+409.090927359"
Jul 10 13:33:05.241906 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:05.241865    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9cb55" podStartSLOduration=1.052103775 podStartE2EDuration="6.241850263s" totalImagesPullingTime="5.189746488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:33:05.241130886 +0000 UTC m=+409.107257738" watchObservedRunningTime="2026-07-10 13:33:05.241850263 +0000 UTC m=+409.107976917"
Jul 10 13:33:06.174405 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:06.174374    2554 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-65h44 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 13:33:06.174755 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:06.174420    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" podUID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 13:33:09.175268 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.175240    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:33:09.374269 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.374233    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"]
Jul 10 13:33:09.377569 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.377546    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.381097 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.381075    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 10 13:33:09.381664 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.381648    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-k2tnf\""
Jul 10 13:33:09.382035 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.382017    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"]
Jul 10 13:33:09.385173 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.385153    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:09.387791 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.387738    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 10 13:33:09.387791 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.387759    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ksrxr\""
Jul 10 13:33:09.389264 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.388754    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"]
Jul 10 13:33:09.397417 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.397398    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"]
Jul 10 13:33:09.478626 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.478559    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbnzw\" (UniqueName: \"kubernetes.io/projected/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-kube-api-access-fbnzw\") pod \"llmisvc-controller-manager-777c6bd7d5-7zgcc\" (UID: \"f6f8f4cc-d87d-4017-960e-9399f98f6e4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:09.478626 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.478593    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01c24155-c090-46d2-8bb0-da82cfa18ff9-cert\") pod \"kserve-controller-manager-d6f4b65b8-7hrxb\" (UID: \"01c24155-c090-46d2-8bb0-da82cfa18ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.478787 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.478641    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7zgcc\" (UID: \"f6f8f4cc-d87d-4017-960e-9399f98f6e4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:09.478787 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.478671    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-597m7\" (UniqueName: \"kubernetes.io/projected/01c24155-c090-46d2-8bb0-da82cfa18ff9-kube-api-access-597m7\") pod \"kserve-controller-manager-d6f4b65b8-7hrxb\" (UID: \"01c24155-c090-46d2-8bb0-da82cfa18ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.579908 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.579878    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbnzw\" (UniqueName: \"kubernetes.io/projected/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-kube-api-access-fbnzw\") pod \"llmisvc-controller-manager-777c6bd7d5-7zgcc\" (UID: \"f6f8f4cc-d87d-4017-960e-9399f98f6e4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:09.579999 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.579914    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01c24155-c090-46d2-8bb0-da82cfa18ff9-cert\") pod \"kserve-controller-manager-d6f4b65b8-7hrxb\" (UID: \"01c24155-c090-46d2-8bb0-da82cfa18ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.579999 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.579963    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7zgcc\" (UID: \"f6f8f4cc-d87d-4017-960e-9399f98f6e4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:09.579999 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.579989    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597m7\" (UniqueName: \"kubernetes.io/projected/01c24155-c090-46d2-8bb0-da82cfa18ff9-kube-api-access-597m7\") pod \"kserve-controller-manager-d6f4b65b8-7hrxb\" (UID: \"01c24155-c090-46d2-8bb0-da82cfa18ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.580148 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:33:09.580088    2554 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 10 13:33:09.580204 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:33:09.580150    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-cert podName:f6f8f4cc-d87d-4017-960e-9399f98f6e4b nodeName:}" failed. No retries permitted until 2026-07-10 13:33:10.080132775 +0000 UTC m=+413.946259408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-cert") pod "llmisvc-controller-manager-777c6bd7d5-7zgcc" (UID: "f6f8f4cc-d87d-4017-960e-9399f98f6e4b") : secret "llmisvc-webhook-server-cert" not found
Jul 10 13:33:09.582264 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.582246    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01c24155-c090-46d2-8bb0-da82cfa18ff9-cert\") pod \"kserve-controller-manager-d6f4b65b8-7hrxb\" (UID: \"01c24155-c090-46d2-8bb0-da82cfa18ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.597291 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.597273    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-597m7\" (UniqueName: \"kubernetes.io/projected/01c24155-c090-46d2-8bb0-da82cfa18ff9-kube-api-access-597m7\") pod \"kserve-controller-manager-d6f4b65b8-7hrxb\" (UID: \"01c24155-c090-46d2-8bb0-da82cfa18ff9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.597490 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.597472    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbnzw\" (UniqueName: \"kubernetes.io/projected/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-kube-api-access-fbnzw\") pod \"llmisvc-controller-manager-777c6bd7d5-7zgcc\" (UID: \"f6f8f4cc-d87d-4017-960e-9399f98f6e4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:09.688845 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.688814    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:09.852763 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:09.852738    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"]
Jul 10 13:33:09.855135 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:33:09.855112    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c24155_c090_46d2_8bb0_da82cfa18ff9.slice/crio-64ee398769c1153866287fdf50009609578e9b4b9db25954eb9173fce2038c37 WatchSource:0}: Error finding container 64ee398769c1153866287fdf50009609578e9b4b9db25954eb9173fce2038c37: Status 404 returned error can't find the container with id 64ee398769c1153866287fdf50009609578e9b4b9db25954eb9173fce2038c37
Jul 10 13:33:10.083729 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:10.083662    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7zgcc\" (UID: \"f6f8f4cc-d87d-4017-960e-9399f98f6e4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:10.085829 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:10.085805    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6f8f4cc-d87d-4017-960e-9399f98f6e4b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7zgcc\" (UID: \"f6f8f4cc-d87d-4017-960e-9399f98f6e4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:10.188365 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:10.188335    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb" event={"ID":"01c24155-c090-46d2-8bb0-da82cfa18ff9","Type":"ContainerStarted","Data":"64ee398769c1153866287fdf50009609578e9b4b9db25954eb9173fce2038c37"}
Jul 10 13:33:10.298921 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:10.298897    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:10.421280 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:10.421256    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"]
Jul 10 13:33:10.423661 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:33:10.423633    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf6f8f4cc_d87d_4017_960e_9399f98f6e4b.slice/crio-cd7b555dca2d78937022bec6bf66dac59d41719fad192b0bb137250f35c4a275 WatchSource:0}: Error finding container cd7b555dca2d78937022bec6bf66dac59d41719fad192b0bb137250f35c4a275: Status 404 returned error can't find the container with id cd7b555dca2d78937022bec6bf66dac59d41719fad192b0bb137250f35c4a275
Jul 10 13:33:11.193125 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:11.193093    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc" event={"ID":"f6f8f4cc-d87d-4017-960e-9399f98f6e4b","Type":"ContainerStarted","Data":"cd7b555dca2d78937022bec6bf66dac59d41719fad192b0bb137250f35c4a275"}
Jul 10 13:33:12.341517 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.341452    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-65h44"]
Jul 10 13:33:12.341964 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.341868    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" podUID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" containerName="discovery" containerID="cri-o://e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631" gracePeriod=30
Jul 10 13:33:12.588505 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.588479    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:33:12.705237 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.705202    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcmxb\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-kube-api-access-gcmxb\") pod \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") "
Jul 10 13:33:12.705406 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.705247    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-dns-cert\") pod \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") "
Jul 10 13:33:12.705406 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.705281    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-token\") pod \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") "
Jul 10 13:33:12.705406 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.705355    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-ca-configmap\") pod \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") "
Jul 10 13:33:12.705562 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.705401    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-kubeconfig\") pod \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") "
Jul 10 13:33:12.705562 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.705464    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-local-certs\") pod \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") "
Jul 10 13:33:12.705562 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.705488    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-cacerts\") pod \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\" (UID: \"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a\") "
Jul 10 13:33:12.706138 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.706101    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" (UID: "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:33:12.708320 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.708277    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-kube-api-access-gcmxb" (OuterVolumeSpecName: "kube-api-access-gcmxb") pod "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" (UID: "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a"). InnerVolumeSpecName "kube-api-access-gcmxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:33:12.708733 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.708558    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" (UID: "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:33:12.709115 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.709075    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-local-certs" (OuterVolumeSpecName: "local-certs") pod "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" (UID: "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:33:12.709605 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.709301    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-cacerts" (OuterVolumeSpecName: "cacerts") pod "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" (UID: "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:33:12.709936 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.709911    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" (UID: "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:33:12.710034 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.709961    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-token" (OuterVolumeSpecName: "istio-token") pod "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" (UID: "926ddb3e-0a81-4b7b-be7b-9a1a1b31094a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:33:12.806848 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.806739    2554 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-kubeconfig\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:33:12.806848 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.806773    2554 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-local-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:33:12.806848 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.806786    2554 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-cacerts\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:33:12.806848 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.806800    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcmxb\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-kube-api-access-gcmxb\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:33:12.807269 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.806815    2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-dns-cert\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:33:12.807269 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.807232    2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-token\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:33:12.807269 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:12.807251    2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a-istio-csr-ca-configmap\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:33:13.207203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.207166    2554 generic.go:356] "Generic (PLEG): container finished" podID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" containerID="e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631" exitCode=0
Jul 10 13:33:13.207386 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.207213    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44"
Jul 10 13:33:13.207386 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.207291    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" event={"ID":"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a","Type":"ContainerDied","Data":"e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631"}
Jul 10 13:33:13.207386 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.207329    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-65h44" event={"ID":"926ddb3e-0a81-4b7b-be7b-9a1a1b31094a","Type":"ContainerDied","Data":"d790f7ccc7f2ffe920e70433a2b2553db093bf4982fd0c3282341c925c7dd727"}
Jul 10 13:33:13.207386 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.207351    2554 scope.go:117] "RemoveContainer" containerID="e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631"
Jul 10 13:33:13.231011 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.230987    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-65h44"]
Jul 10 13:33:13.236302 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.236282    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-65h44"]
Jul 10 13:33:13.747732 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.747565    2554 scope.go:117] "RemoveContainer" containerID="e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631"
Jul 10 13:33:13.748003 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:33:13.747928    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631\": container with ID starting with e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631 not found: ID does not exist" containerID="e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631"
Jul 10 13:33:13.748003 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:13.747958    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631"} err="failed to get container status \"e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631\": rpc error: code = NotFound desc = could not find container \"e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631\": container with ID starting with e00e167d1591cd1a51c1890eb3d5584d384f1c8963ded747d1dd24130b3a4631 not found: ID does not exist"
Jul 10 13:33:14.213855 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:14.213800    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb" event={"ID":"01c24155-c090-46d2-8bb0-da82cfa18ff9","Type":"ContainerStarted","Data":"2006743a3443032c2d4736d34d846ad9d3a84df4648121af9d2ad639fa4cd9b0"}
Jul 10 13:33:14.214029 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:14.213891    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:14.215407 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:14.215386    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc" event={"ID":"f6f8f4cc-d87d-4017-960e-9399f98f6e4b","Type":"ContainerStarted","Data":"f905d3f082c6ce334d789aa120e489df446b5bc7f619d03925b1e0afc103b4f2"}
Jul 10 13:33:14.215523 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:14.215509    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:14.232129 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:14.232082    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb" podStartSLOduration=1.332301102 podStartE2EDuration="5.232069674s" totalImagesPullingTime="3.899768572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:33:14.231182012 +0000 UTC m=+418.097308667" watchObservedRunningTime="2026-07-10 13:33:14.232069674 +0000 UTC m=+418.098196365"
Jul 10 13:33:14.249327 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:14.249291    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc" podStartSLOduration=1.913532877 podStartE2EDuration="5.24927949s" totalImagesPullingTime="3.335746613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:33:14.248574991 +0000 UTC m=+418.114701644" watchObservedRunningTime="2026-07-10 13:33:14.24927949 +0000 UTC m=+418.115406143"
Jul 10 13:33:14.663600 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:14.663530    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" path="/var/lib/kubelet/pods/926ddb3e-0a81-4b7b-be7b-9a1a1b31094a/volumes"
Jul 10 13:33:16.175507 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:16.175479    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-24j8m"
Jul 10 13:33:16.177646 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:16.177622    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9cb55"
Jul 10 13:33:45.221743 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:45.221666    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7zgcc"
Jul 10 13:33:45.224734 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:45.224715    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7hrxb"
Jul 10 13:33:59.243107 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.243074    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4pmjj"]
Jul 10 13:33:59.243463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.243334    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" containerName="discovery"
Jul 10 13:33:59.243463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.243344    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" containerName="discovery"
Jul 10 13:33:59.243463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.243427    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="926ddb3e-0a81-4b7b-be7b-9a1a1b31094a" containerName="discovery"
Jul 10 13:33:59.247908 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.247886    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj"
Jul 10 13:33:59.250695 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.250676    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 13:33:59.251754 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.251719    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-tdnr5\""
Jul 10 13:33:59.251754 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.251740    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 13:33:59.256322 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.256298    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4pmjj"]
Jul 10 13:33:59.318173 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.318150    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqclk\" (UniqueName: \"kubernetes.io/projected/c17376f0-acd6-48a7-8f52-f88aa77a20ee-kube-api-access-nqclk\") pod \"kuadrant-operator-catalog-4pmjj\" (UID: \"c17376f0-acd6-48a7-8f52-f88aa77a20ee\") " pod="kuadrant-system/kuadrant-operator-catalog-4pmjj"
Jul 10 13:33:59.419039 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.419021    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqclk\" (UniqueName: \"kubernetes.io/projected/c17376f0-acd6-48a7-8f52-f88aa77a20ee-kube-api-access-nqclk\") pod \"kuadrant-operator-catalog-4pmjj\" (UID: \"c17376f0-acd6-48a7-8f52-f88aa77a20ee\") " pod="kuadrant-system/kuadrant-operator-catalog-4pmjj"
Jul 10 13:33:59.428977 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.428952    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqclk\" (UniqueName: \"kubernetes.io/projected/c17376f0-acd6-48a7-8f52-f88aa77a20ee-kube-api-access-nqclk\") pod \"kuadrant-operator-catalog-4pmjj\" (UID: \"c17376f0-acd6-48a7-8f52-f88aa77a20ee\") " pod="kuadrant-system/kuadrant-operator-catalog-4pmjj"
Jul 10 13:33:59.559315 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.559246    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj"
Jul 10 13:33:59.599023 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.598997    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4pmjj"]
Jul 10 13:33:59.700086 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.700064    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4pmjj"]
Jul 10 13:33:59.702364 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:33:59.702336    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17376f0_acd6_48a7_8f52_f88aa77a20ee.slice/crio-d93e79b64f9448266b6f8f645feff516f9a6b054e3c7c730c95aba28252dfae8 WatchSource:0}: Error finding container d93e79b64f9448266b6f8f645feff516f9a6b054e3c7c730c95aba28252dfae8: Status 404 returned error can't find the container with id d93e79b64f9448266b6f8f645feff516f9a6b054e3c7c730c95aba28252dfae8
Jul 10 13:33:59.806786 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.806763    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-896m6"]
Jul 10 13:33:59.809570 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.809533    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:33:59.817834 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.817799    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-896m6"]
Jul 10 13:33:59.922039 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:33:59.922016    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvvkq\" (UniqueName: \"kubernetes.io/projected/7ff0a59a-e655-4860-8a76-e18108eac3c0-kube-api-access-dvvkq\") pod \"kuadrant-operator-catalog-896m6\" (UID: \"7ff0a59a-e655-4860-8a76-e18108eac3c0\") " pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:00.023204 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:00.023175    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvvkq\" (UniqueName: \"kubernetes.io/projected/7ff0a59a-e655-4860-8a76-e18108eac3c0-kube-api-access-dvvkq\") pod \"kuadrant-operator-catalog-896m6\" (UID: \"7ff0a59a-e655-4860-8a76-e18108eac3c0\") " pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:00.033199 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:00.033175    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvvkq\" (UniqueName: \"kubernetes.io/projected/7ff0a59a-e655-4860-8a76-e18108eac3c0-kube-api-access-dvvkq\") pod \"kuadrant-operator-catalog-896m6\" (UID: \"7ff0a59a-e655-4860-8a76-e18108eac3c0\") " pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:00.118974 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:00.118914    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:00.245551 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:00.245530    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-896m6"]
Jul 10 13:34:00.247746 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:34:00.247720    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff0a59a_e655_4860_8a76_e18108eac3c0.slice/crio-f8567ffd0efc274e5395332d099a76baf130cfa0b0311eaa09ac27e841d81409 WatchSource:0}: Error finding container f8567ffd0efc274e5395332d099a76baf130cfa0b0311eaa09ac27e841d81409: Status 404 returned error can't find the container with id f8567ffd0efc274e5395332d099a76baf130cfa0b0311eaa09ac27e841d81409
Jul 10 13:34:00.405441 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:00.405376    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj" event={"ID":"c17376f0-acd6-48a7-8f52-f88aa77a20ee","Type":"ContainerStarted","Data":"d93e79b64f9448266b6f8f645feff516f9a6b054e3c7c730c95aba28252dfae8"}
Jul 10 13:34:00.406481 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:00.406459    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-896m6" event={"ID":"7ff0a59a-e655-4860-8a76-e18108eac3c0","Type":"ContainerStarted","Data":"f8567ffd0efc274e5395332d099a76baf130cfa0b0311eaa09ac27e841d81409"}
Jul 10 13:34:02.416753 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.416664    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj" event={"ID":"c17376f0-acd6-48a7-8f52-f88aa77a20ee","Type":"ContainerStarted","Data":"bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5"}
Jul 10 13:34:02.417195 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.416721    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj" podUID="c17376f0-acd6-48a7-8f52-f88aa77a20ee" containerName="registry-server" containerID="cri-o://bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5" gracePeriod=2
Jul 10 13:34:02.418188 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.418154    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-896m6" event={"ID":"7ff0a59a-e655-4860-8a76-e18108eac3c0","Type":"ContainerStarted","Data":"134d7463a2eee7d6ce922c8d8796ad38d510d3ee43c2cf29e66bab7d3931dd64"}
Jul 10 13:34:02.434344 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.434306    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj" podStartSLOduration=1.149052732 podStartE2EDuration="3.434291238s" totalImagesPullingTime="2.285238506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:34:02.432596786 +0000 UTC m=+466.298723440" watchObservedRunningTime="2026-07-10 13:34:02.434291238 +0000 UTC m=+466.300417891"
Jul 10 13:34:02.449064 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.449031    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-896m6" podStartSLOduration=1.694687204 podStartE2EDuration="3.449021331s" totalImagesPullingTime="1.754334127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:34:02.448882152 +0000 UTC m=+466.315008806" watchObservedRunningTime="2026-07-10 13:34:02.449021331 +0000 UTC m=+466.315147986"
Jul 10 13:34:02.653261 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.653241    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj"
Jul 10 13:34:02.748801 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.748769    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqclk\" (UniqueName: \"kubernetes.io/projected/c17376f0-acd6-48a7-8f52-f88aa77a20ee-kube-api-access-nqclk\") pod \"c17376f0-acd6-48a7-8f52-f88aa77a20ee\" (UID: \"c17376f0-acd6-48a7-8f52-f88aa77a20ee\") "
Jul 10 13:34:02.750814 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.750795    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17376f0-acd6-48a7-8f52-f88aa77a20ee-kube-api-access-nqclk" (OuterVolumeSpecName: "kube-api-access-nqclk") pod "c17376f0-acd6-48a7-8f52-f88aa77a20ee" (UID: "c17376f0-acd6-48a7-8f52-f88aa77a20ee"). InnerVolumeSpecName "kube-api-access-nqclk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:34:02.849508 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:02.849482    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqclk\" (UniqueName: \"kubernetes.io/projected/c17376f0-acd6-48a7-8f52-f88aa77a20ee-kube-api-access-nqclk\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:34:03.423657 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.423564    2554 generic.go:356] "Generic (PLEG): container finished" podID="c17376f0-acd6-48a7-8f52-f88aa77a20ee" containerID="bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5" exitCode=0
Jul 10 13:34:03.423657 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.423614    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj"
Jul 10 13:34:03.424164 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.423653    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj" event={"ID":"c17376f0-acd6-48a7-8f52-f88aa77a20ee","Type":"ContainerDied","Data":"bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5"}
Jul 10 13:34:03.424164 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.423731    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4pmjj" event={"ID":"c17376f0-acd6-48a7-8f52-f88aa77a20ee","Type":"ContainerDied","Data":"d93e79b64f9448266b6f8f645feff516f9a6b054e3c7c730c95aba28252dfae8"}
Jul 10 13:34:03.424164 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.423756    2554 scope.go:117] "RemoveContainer" containerID="bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5"
Jul 10 13:34:03.432694 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.432678    2554 scope.go:117] "RemoveContainer" containerID="bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5"
Jul 10 13:34:03.432979 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:34:03.432961    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5\": container with ID starting with bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5 not found: ID does not exist" containerID="bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5"
Jul 10 13:34:03.433048 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.432986    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5"} err="failed to get container status \"bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5\": rpc error: code = NotFound desc = could not find container \"bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5\": container with ID starting with bf7a185e040830121de8370156f075e885a2ea460fdc9ace5d0946139d753bf5 not found: ID does not exist"
Jul 10 13:34:03.447460 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.447441    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4pmjj"]
Jul 10 13:34:03.452133 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:03.452116    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4pmjj"]
Jul 10 13:34:04.662998 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:04.662971    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c17376f0-acd6-48a7-8f52-f88aa77a20ee" path="/var/lib/kubelet/pods/c17376f0-acd6-48a7-8f52-f88aa77a20ee/volumes"
Jul 10 13:34:10.119687 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:10.119653    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:10.119687 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:10.119707    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:10.141484 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:10.141461    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:10.474699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:10.474673    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-896m6"
Jul 10 13:34:36.795128 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.795096    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"]
Jul 10 13:34:36.795560 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.795389    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c17376f0-acd6-48a7-8f52-f88aa77a20ee" containerName="registry-server"
Jul 10 13:34:36.795560 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.795401    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17376f0-acd6-48a7-8f52-f88aa77a20ee" containerName="registry-server"
Jul 10 13:34:36.795560 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.795461    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c17376f0-acd6-48a7-8f52-f88aa77a20ee" containerName="registry-server"
Jul 10 13:34:36.803452 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.803433    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"
Jul 10 13:34:36.806656 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.806632    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7hkb5\""
Jul 10 13:34:36.806656 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.806649    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 10 13:34:36.810693 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.810675    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"]
Jul 10 13:34:36.887111 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.887090    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvtqv\" (UniqueName: \"kubernetes.io/projected/49eae6c4-a529-412e-93ca-899af3234504-kube-api-access-kvtqv\") pod \"dns-operator-controller-manager-864dd78c4c-cvdt5\" (UID: \"49eae6c4-a529-412e-93ca-899af3234504\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"
Jul 10 13:34:36.988255 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:36.988233    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvtqv\" (UniqueName: \"kubernetes.io/projected/49eae6c4-a529-412e-93ca-899af3234504-kube-api-access-kvtqv\") pod \"dns-operator-controller-manager-864dd78c4c-cvdt5\" (UID: \"49eae6c4-a529-412e-93ca-899af3234504\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"
Jul 10 13:34:37.001524 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:37.001499    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvtqv\" (UniqueName: \"kubernetes.io/projected/49eae6c4-a529-412e-93ca-899af3234504-kube-api-access-kvtqv\") pod \"dns-operator-controller-manager-864dd78c4c-cvdt5\" (UID: \"49eae6c4-a529-412e-93ca-899af3234504\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"
Jul 10 13:34:37.114057 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:37.114008    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"
Jul 10 13:34:37.246090 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:37.246060    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"]
Jul 10 13:34:37.248837 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:34:37.248787    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49eae6c4_a529_412e_93ca_899af3234504.slice/crio-8a1727bd103b3d5b17ff635ac43758cdce7698f0cba8ec5c3f0b96d015a2735c WatchSource:0}: Error finding container 8a1727bd103b3d5b17ff635ac43758cdce7698f0cba8ec5c3f0b96d015a2735c: Status 404 returned error can't find the container with id 8a1727bd103b3d5b17ff635ac43758cdce7698f0cba8ec5c3f0b96d015a2735c
Jul 10 13:34:37.560752 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:37.560721    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5" event={"ID":"49eae6c4-a529-412e-93ca-899af3234504","Type":"ContainerStarted","Data":"8a1727bd103b3d5b17ff635ac43758cdce7698f0cba8ec5c3f0b96d015a2735c"}
Jul 10 13:34:40.581732 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:40.581700    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5" event={"ID":"49eae6c4-a529-412e-93ca-899af3234504","Type":"ContainerStarted","Data":"574fa3eaee76c0615449dd07e1b8aa100f1b04154e82551df72b7336e94f7083"}
Jul 10 13:34:40.582105 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:40.581790    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"
Jul 10 13:34:40.603052 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:40.603011    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5" podStartSLOduration=2.19875822 podStartE2EDuration="4.602997606s" totalImagesPullingTime="2.404239386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:34:40.601178718 +0000 UTC m=+504.467305372" watchObservedRunningTime="2026-07-10 13:34:40.602997606 +0000 UTC m=+504.469124259"
Jul 10 13:34:41.901005 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:41.900971    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"]
Jul 10 13:34:41.904113 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:41.904098    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:41.907833 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:41.907802    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cgx88\""
Jul 10 13:34:41.917015 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:41.916995    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"]
Jul 10 13:34:42.026832 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.026792    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zb7p\" (UniqueName: \"kubernetes.io/projected/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-kube-api-access-8zb7p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:42.027008 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.026845    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:42.127735 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.127701    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zb7p\" (UniqueName: \"kubernetes.io/projected/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-kube-api-access-8zb7p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:42.127888 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.127747    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:42.128093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.128077    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:42.139096 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.139074    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zb7p\" (UniqueName: \"kubernetes.io/projected/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-kube-api-access-8zb7p\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:42.214064 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.214048    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:42.355666 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:34:42.355635    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3daae0c1_e4a0_48d0_bd66_dcfcde0fad2d.slice/crio-3733105ea44fdf6074cff5f16377004a44881064d042f9570b11593b556f0108 WatchSource:0}: Error finding container 3733105ea44fdf6074cff5f16377004a44881064d042f9570b11593b556f0108: Status 404 returned error can't find the container with id 3733105ea44fdf6074cff5f16377004a44881064d042f9570b11593b556f0108
Jul 10 13:34:42.357033 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.356806    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"]
Jul 10 13:34:42.589777 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:42.589715    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f" event={"ID":"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d","Type":"ContainerStarted","Data":"3733105ea44fdf6074cff5f16377004a44881064d042f9570b11593b556f0108"}
Jul 10 13:34:45.778341 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:45.778308    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"]
Jul 10 13:34:45.781525 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:45.781510    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"
Jul 10 13:34:45.784190 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:45.784172    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-flzjx\""
Jul 10 13:34:45.792956 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:45.792935    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"]
Jul 10 13:34:45.954914 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:45.954885    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk5xk\" (UniqueName: \"kubernetes.io/projected/71f8461c-559b-449f-803d-362f7bc02571-kube-api-access-tk5xk\") pod \"authorino-operator-75cc8d5cb7-gdl77\" (UID: \"71f8461c-559b-449f-803d-362f7bc02571\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"
Jul 10 13:34:46.055525 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:46.055453    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk5xk\" (UniqueName: \"kubernetes.io/projected/71f8461c-559b-449f-803d-362f7bc02571-kube-api-access-tk5xk\") pod \"authorino-operator-75cc8d5cb7-gdl77\" (UID: \"71f8461c-559b-449f-803d-362f7bc02571\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"
Jul 10 13:34:46.068961 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:46.068935    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk5xk\" (UniqueName: \"kubernetes.io/projected/71f8461c-559b-449f-803d-362f7bc02571-kube-api-access-tk5xk\") pod \"authorino-operator-75cc8d5cb7-gdl77\" (UID: \"71f8461c-559b-449f-803d-362f7bc02571\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"
Jul 10 13:34:46.091785 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:46.091767    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"
Jul 10 13:34:46.228472 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:46.228447    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"]
Jul 10 13:34:46.230440 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:34:46.230417    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f8461c_559b_449f_803d_362f7bc02571.slice/crio-29aa2b6d9e9df2adfe87d95434a00dd3ba6c6b21ed1f614ed7413f37e41dc403 WatchSource:0}: Error finding container 29aa2b6d9e9df2adfe87d95434a00dd3ba6c6b21ed1f614ed7413f37e41dc403: Status 404 returned error can't find the container with id 29aa2b6d9e9df2adfe87d95434a00dd3ba6c6b21ed1f614ed7413f37e41dc403
Jul 10 13:34:46.606048 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:46.606014    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77" event={"ID":"71f8461c-559b-449f-803d-362f7bc02571","Type":"ContainerStarted","Data":"29aa2b6d9e9df2adfe87d95434a00dd3ba6c6b21ed1f614ed7413f37e41dc403"}
Jul 10 13:34:48.690500 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:48.690468    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"]
Jul 10 13:34:48.698924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:48.698902    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:34:48.701679 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:48.701655    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wgdcd\""
Jul 10 13:34:48.711724 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:48.709840    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"]
Jul 10 13:34:48.874923 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:48.874892    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jjsn\" (UniqueName: \"kubernetes.io/projected/27c92c25-1393-46b5-9cf8-c52a39098880-kube-api-access-5jjsn\") pod \"limitador-operator-controller-manager-859d7b78bd-tlmxw\" (UID: \"27c92c25-1393-46b5-9cf8-c52a39098880\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:34:48.976227 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:48.976157    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jjsn\" (UniqueName: \"kubernetes.io/projected/27c92c25-1393-46b5-9cf8-c52a39098880-kube-api-access-5jjsn\") pod \"limitador-operator-controller-manager-859d7b78bd-tlmxw\" (UID: \"27c92c25-1393-46b5-9cf8-c52a39098880\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:34:48.988757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:48.988720    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jjsn\" (UniqueName: \"kubernetes.io/projected/27c92c25-1393-46b5-9cf8-c52a39098880-kube-api-access-5jjsn\") pod \"limitador-operator-controller-manager-859d7b78bd-tlmxw\" (UID: \"27c92c25-1393-46b5-9cf8-c52a39098880\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:34:49.015958 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.015911    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:34:49.386175 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.386153    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"]
Jul 10 13:34:49.389255 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:34:49.389231    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c92c25_1393_46b5_9cf8_c52a39098880.slice/crio-3635d5234e03820e35923a7de5422ac6c60478029c7ddda1925ffe38b26c140e WatchSource:0}: Error finding container 3635d5234e03820e35923a7de5422ac6c60478029c7ddda1925ffe38b26c140e: Status 404 returned error can't find the container with id 3635d5234e03820e35923a7de5422ac6c60478029c7ddda1925ffe38b26c140e
Jul 10 13:34:49.621610 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.621528    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77" event={"ID":"71f8461c-559b-449f-803d-362f7bc02571","Type":"ContainerStarted","Data":"bfe477c57fd38dd7817205932270ef3ffbd8d602039d12d66ae8c3b54f40b217"}
Jul 10 13:34:49.621610 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.621590    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"
Jul 10 13:34:49.623097 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.623059    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f" event={"ID":"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d","Type":"ContainerStarted","Data":"2e7b0da867b3bca5868ff5616b28e1354a055f6e852a6a5ddda910eddb35d4a9"}
Jul 10 13:34:49.623221 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.623130    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:34:49.624219 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.624199    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw" event={"ID":"27c92c25-1393-46b5-9cf8-c52a39098880","Type":"ContainerStarted","Data":"3635d5234e03820e35923a7de5422ac6c60478029c7ddda1925ffe38b26c140e"}
Jul 10 13:34:49.642722 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.642684    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77" podStartSLOduration=1.575986678 podStartE2EDuration="4.642672015s" totalImagesPullingTime="3.066685337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:34:49.642069483 +0000 UTC m=+513.508196134" watchObservedRunningTime="2026-07-10 13:34:49.642672015 +0000 UTC m=+513.508798668"
Jul 10 13:34:49.668610 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:49.668568    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f" podStartSLOduration=1.770054074 podStartE2EDuration="8.668553968s" totalImagesPullingTime="6.898499894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:34:49.666618844 +0000 UTC m=+513.532745498" watchObservedRunningTime="2026-07-10 13:34:49.668553968 +0000 UTC m=+513.534680623"
Jul 10 13:34:51.587381 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:51.587344    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cvdt5"
Jul 10 13:34:51.635632 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:51.635593    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw" event={"ID":"27c92c25-1393-46b5-9cf8-c52a39098880","Type":"ContainerStarted","Data":"0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41"}
Jul 10 13:34:51.635745 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:51.635671    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:34:51.656932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:34:51.656890    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw" podStartSLOduration=1.5955809090000002 podStartE2EDuration="3.656875653s" totalImagesPullingTime="2.061294744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:34:51.655853751 +0000 UTC m=+515.521980406" watchObservedRunningTime="2026-07-10 13:34:51.656875653 +0000 UTC m=+515.523002307"
Jul 10 13:35:00.631792 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:00.631764    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gdl77"
Jul 10 13:35:00.632173 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:00.631851    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:35:02.321069 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.321039    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"]
Jul 10 13:35:02.324502 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.324485    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.337322 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.337299    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"]
Jul 10 13:35:02.373014 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.372990    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.373112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.373031    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l6nt\" (UniqueName: \"kubernetes.io/projected/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-kube-api-access-9l6nt\") pod \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.423838 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.423804    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"]
Jul 10 13:35:02.424031 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.424009    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-9l6nt], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1"
Jul 10 13:35:02.474023 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.474000    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.474135 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.474040    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l6nt\" (UniqueName: \"kubernetes.io/projected/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-kube-api-access-9l6nt\") pod \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.474365 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.474343    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.487153 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.487131    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l6nt\" (UniqueName: \"kubernetes.io/projected/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-kube-api-access-9l6nt\") pod \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.546636 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.546614    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"]
Jul 10 13:35:02.554260 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.554240    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"]
Jul 10 13:35:02.565604 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.565575    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"]
Jul 10 13:35:02.565769 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.565749    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" containerName="manager" containerID="cri-o://2e7b0da867b3bca5868ff5616b28e1354a055f6e852a6a5ddda910eddb35d4a9" gracePeriod=2
Jul 10 13:35:02.574122 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.574056    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"]
Jul 10 13:35:02.577923 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.577894    2554 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\": pods \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:35:02.588939 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.588918    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"]
Jul 10 13:35:02.589200 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.589189    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" containerName="manager"
Jul 10 13:35:02.589244 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.589201    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" containerName="manager"
Jul 10 13:35:02.589278 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.589254    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" containerName="manager"
Jul 10 13:35:02.592191 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.592173    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:02.594498 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.594472    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:35:02.607904 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.607884    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"]
Jul 10 13:35:02.615712 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.615693    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"]
Jul 10 13:35:02.615900 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.615880    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" containerName="manager" containerID="cri-o://0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41" gracePeriod=2
Jul 10 13:35:02.617452 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.617423    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:35:02.624134 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.624111    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"]
Jul 10 13:35:02.629229 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.629199    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:35:02.633072 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.633043    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlmxw\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:35:02.637010 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.636977    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"]
Jul 10 13:35:02.637136 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.637084    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:02.639395 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.639374    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"]
Jul 10 13:35:02.645983 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.645962    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"]
Jul 10 13:35:02.646490 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.646470    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" containerName="manager"
Jul 10 13:35:02.646622 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.646600    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" containerName="manager"
Jul 10 13:35:02.646755 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.646710    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" containerName="manager"
Jul 10 13:35:02.649610 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.649590    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"
Jul 10 13:35:02.665024 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.665003    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"]
Jul 10 13:35:02.665410 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.665387    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlmxw\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:35:02.667408 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.667382    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4cm5f\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:35:02.675350 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.675323    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8789a79-0a3f-4365-9328-7833511eb136-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f6smt\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:02.675452 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.675352    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79991ff7-31df-4e6c-bbd7-f0a834044ce7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gqswk\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:02.675452 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.675394    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gjq2\" (UniqueName: \"kubernetes.io/projected/23ade6a6-8418-44e0-9ee3-f133b75cbeb7-kube-api-access-7gjq2\") pod \"limitador-operator-controller-manager-859d7b78bd-5srn4\" (UID: \"23ade6a6-8418-44e0-9ee3-f133b75cbeb7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"
Jul 10 13:35:02.675452 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.675443    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpq69\" (UniqueName: \"kubernetes.io/projected/79991ff7-31df-4e6c-bbd7-f0a834044ce7-kube-api-access-fpq69\") pod \"kuadrant-operator-controller-manager-86878c769f-gqswk\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:02.675550 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.675468    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqtp8\" (UniqueName: \"kubernetes.io/projected/a8789a79-0a3f-4365-9328-7833511eb136-kube-api-access-hqtp8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f6smt\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:02.686148 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.686125    2554 generic.go:356] "Generic (PLEG): container finished" podID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" containerID="2e7b0da867b3bca5868ff5616b28e1354a055f6e852a6a5ddda910eddb35d4a9" exitCode=0
Jul 10 13:35:02.686216 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.686197    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.729394 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.729378    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.731689 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.731667    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.776692 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776661    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-extensions-socket-volume\") pod \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") "
Jul 10 13:35:02.776891 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776723    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l6nt\" (UniqueName: \"kubernetes.io/projected/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-kube-api-access-9l6nt\") pod \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\" (UID: \"5eefa17b-ca65-47a4-a8ce-495e29bb00f1\") "
Jul 10 13:35:02.776891 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776830    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gjq2\" (UniqueName: \"kubernetes.io/projected/23ade6a6-8418-44e0-9ee3-f133b75cbeb7-kube-api-access-7gjq2\") pod \"limitador-operator-controller-manager-859d7b78bd-5srn4\" (UID: \"23ade6a6-8418-44e0-9ee3-f133b75cbeb7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"
Jul 10 13:35:02.776891 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776873    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpq69\" (UniqueName: \"kubernetes.io/projected/79991ff7-31df-4e6c-bbd7-f0a834044ce7-kube-api-access-fpq69\") pod \"kuadrant-operator-controller-manager-86878c769f-gqswk\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:02.777083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776914    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqtp8\" (UniqueName: \"kubernetes.io/projected/a8789a79-0a3f-4365-9328-7833511eb136-kube-api-access-hqtp8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f6smt\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:02.777083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776936    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5eefa17b-ca65-47a4-a8ce-495e29bb00f1" (UID: "5eefa17b-ca65-47a4-a8ce-495e29bb00f1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:35:02.777083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776962    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8789a79-0a3f-4365-9328-7833511eb136-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f6smt\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:02.777083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.776987    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79991ff7-31df-4e6c-bbd7-f0a834044ce7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gqswk\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:02.777083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.777026    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-extensions-socket-volume\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:02.777615 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.777381    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79991ff7-31df-4e6c-bbd7-f0a834044ce7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gqswk\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:02.777766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.777744    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8789a79-0a3f-4365-9328-7833511eb136-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f6smt\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:02.779123 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.779102    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-kube-api-access-9l6nt" (OuterVolumeSpecName: "kube-api-access-9l6nt") pod "5eefa17b-ca65-47a4-a8ce-495e29bb00f1" (UID: "5eefa17b-ca65-47a4-a8ce-495e29bb00f1"). InnerVolumeSpecName "kube-api-access-9l6nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:35:02.788390 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.788195    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gjq2\" (UniqueName: \"kubernetes.io/projected/23ade6a6-8418-44e0-9ee3-f133b75cbeb7-kube-api-access-7gjq2\") pod \"limitador-operator-controller-manager-859d7b78bd-5srn4\" (UID: \"23ade6a6-8418-44e0-9ee3-f133b75cbeb7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"
Jul 10 13:35:02.788390 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.788260    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpq69\" (UniqueName: \"kubernetes.io/projected/79991ff7-31df-4e6c-bbd7-f0a834044ce7-kube-api-access-fpq69\") pod \"kuadrant-operator-controller-manager-86878c769f-gqswk\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:02.790028 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.790008    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqtp8\" (UniqueName: \"kubernetes.io/projected/a8789a79-0a3f-4365-9328-7833511eb136-kube-api-access-hqtp8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f6smt\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:02.855522 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.855496    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:35:02.858139 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.858110    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.861621 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.861604    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:35:02.863831 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:02.863789    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:02.878120 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.878104    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jjsn\" (UniqueName: \"kubernetes.io/projected/27c92c25-1393-46b5-9cf8-c52a39098880-kube-api-access-5jjsn\") pod \"27c92c25-1393-46b5-9cf8-c52a39098880\" (UID: \"27c92c25-1393-46b5-9cf8-c52a39098880\") "
Jul 10 13:35:02.878196 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.878134    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zb7p\" (UniqueName: \"kubernetes.io/projected/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-kube-api-access-8zb7p\") pod \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") "
Jul 10 13:35:02.878233 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.878197    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-extensions-socket-volume\") pod \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\" (UID: \"3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d\") "
Jul 10 13:35:02.878342 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.878322    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l6nt\" (UniqueName: \"kubernetes.io/projected/5eefa17b-ca65-47a4-a8ce-495e29bb00f1-kube-api-access-9l6nt\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:02.878722 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.878700    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" (UID: "3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:35:02.880075 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.880048    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c92c25-1393-46b5-9cf8-c52a39098880-kube-api-access-5jjsn" (OuterVolumeSpecName: "kube-api-access-5jjsn") pod "27c92c25-1393-46b5-9cf8-c52a39098880" (UID: "27c92c25-1393-46b5-9cf8-c52a39098880"). InnerVolumeSpecName "kube-api-access-5jjsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:35:02.880147 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.880121    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-kube-api-access-8zb7p" (OuterVolumeSpecName: "kube-api-access-8zb7p") pod "3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" (UID: "3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d"). InnerVolumeSpecName "kube-api-access-8zb7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:35:02.978763 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.978741    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-extensions-socket-volume\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:02.978763 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.978762    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jjsn\" (UniqueName: \"kubernetes.io/projected/27c92c25-1393-46b5-9cf8-c52a39098880-kube-api-access-5jjsn\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:02.978904 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:02.978770    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zb7p\" (UniqueName: \"kubernetes.io/projected/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d-kube-api-access-8zb7p\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:03.011711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.011684    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:03.018126 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.018109    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:03.026121 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.026103    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"
Jul 10 13:35:03.201383 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.201346    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"]
Jul 10 13:35:03.207491 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:35:03.207449    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79991ff7_31df_4e6c_bbd7_f0a834044ce7.slice/crio-daec6ee277e4fa794558c045044da50d551e7045ab053632c0c70d860e8cb90d WatchSource:0}: Error finding container daec6ee277e4fa794558c045044da50d551e7045ab053632c0c70d860e8cb90d: Status 404 returned error can't find the container with id daec6ee277e4fa794558c045044da50d551e7045ab053632c0c70d860e8cb90d
Jul 10 13:35:03.208848 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.208797    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"]
Jul 10 13:35:03.210500 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.210479    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"]
Jul 10 13:35:03.212006 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:35:03.211987    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8789a79_0a3f_4365_9328_7833511eb136.slice/crio-9c77d461ed228c127c7042f72aaada9c5009a3457c280cef3eb5b9fc36afb0c0 WatchSource:0}: Error finding container 9c77d461ed228c127c7042f72aaada9c5009a3457c280cef3eb5b9fc36afb0c0: Status 404 returned error can't find the container with id 9c77d461ed228c127c7042f72aaada9c5009a3457c280cef3eb5b9fc36afb0c0
Jul 10 13:35:03.212954 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:35:03.212929    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ade6a6_8418_44e0_9ee3_f133b75cbeb7.slice/crio-0a0656d782cd62cd7283597abd36d9f29b363e53454c13d482651dc86394a248 WatchSource:0}: Error finding container 0a0656d782cd62cd7283597abd36d9f29b363e53454c13d482651dc86394a248: Status 404 returned error can't find the container with id 0a0656d782cd62cd7283597abd36d9f29b363e53454c13d482651dc86394a248
Jul 10 13:35:03.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.692975    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4cm5f"
Jul 10 13:35:03.693004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.692987    2554 scope.go:117] "RemoveContainer" containerID="2e7b0da867b3bca5868ff5616b28e1354a055f6e852a6a5ddda910eddb35d4a9"
Jul 10 13:35:03.694440 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.694415    2554 generic.go:356] "Generic (PLEG): container finished" podID="27c92c25-1393-46b5-9cf8-c52a39098880" containerID="0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41" exitCode=0
Jul 10 13:35:03.694527 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.694464    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:35:03.695708 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:03.695623    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:03.696240 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.696214    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4" event={"ID":"23ade6a6-8418-44e0-9ee3-f133b75cbeb7","Type":"ContainerStarted","Data":"243dd850286d95d22eec6f6dd7819647eb0fd4984fb96d84f2edb03bd20abf7b"}
Jul 10 13:35:03.696357 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.696274    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"
Jul 10 13:35:03.696357 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.696298    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4" event={"ID":"23ade6a6-8418-44e0-9ee3-f133b75cbeb7","Type":"ContainerStarted","Data":"0a0656d782cd62cd7283597abd36d9f29b363e53454c13d482651dc86394a248"}
Jul 10 13:35:03.697850 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:03.697760    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:03.698310 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.698282    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt" event={"ID":"a8789a79-0a3f-4365-9328-7833511eb136","Type":"ContainerStarted","Data":"3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c"}
Jul 10 13:35:03.698422 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.698343    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt" event={"ID":"a8789a79-0a3f-4365-9328-7833511eb136","Type":"ContainerStarted","Data":"9c77d461ed228c127c7042f72aaada9c5009a3457c280cef3eb5b9fc36afb0c0"}
Jul 10 13:35:03.698422 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.698373    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:03.700189 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.700172    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:03.700311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.700171    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk" event={"ID":"79991ff7-31df-4e6c-bbd7-f0a834044ce7","Type":"ContainerStarted","Data":"0637f55a86ca5ed80bac550fb1ff2b4cab1fd6efdda6fc8fe4c787c7f2d8a55e"}
Jul 10 13:35:03.700311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.700228    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk" event={"ID":"79991ff7-31df-4e6c-bbd7-f0a834044ce7","Type":"ContainerStarted","Data":"daec6ee277e4fa794558c045044da50d551e7045ab053632c0c70d860e8cb90d"}
Jul 10 13:35:03.700311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.700273    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:03.703605 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.703586    2554 scope.go:117] "RemoveContainer" containerID="0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41"
Jul 10 13:35:03.711710 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.711693    2554 scope.go:117] "RemoveContainer" containerID="0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41"
Jul 10 13:35:03.711952 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:03.711936    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41\": container with ID starting with 0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41 not found: ID does not exist" containerID="0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41"
Jul 10 13:35:03.712006 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.711960    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41"} err="failed to get container status \"0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41\": rpc error: code = NotFound desc = could not find container \"0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41\": container with ID starting with 0c02bbaf69d6cdd514e948661162dd44b49fca559d09c2f11a7ccd3877ae9f41 not found: ID does not exist"
Jul 10 13:35:03.717738 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.717698    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4" podStartSLOduration=1.717683149 podStartE2EDuration="1.717683149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:35:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:35:03.717669763 +0000 UTC m=+527.583796418" watchObservedRunningTime="2026-07-10 13:35:03.717683149 +0000 UTC m=+527.583809804"
Jul 10 13:35:03.720229 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:03.720202    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:03.740668 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.740632    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk" podStartSLOduration=1.740618999 podStartE2EDuration="1.740618999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:35:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:35:03.739594748 +0000 UTC m=+527.605721406" watchObservedRunningTime="2026-07-10 13:35:03.740618999 +0000 UTC m=+527.606745652"
Jul 10 13:35:03.743166 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:03.743146    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jghf4\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jghf4"
Jul 10 13:35:03.770552 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:03.770486    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt" podStartSLOduration=1.770473713 podStartE2EDuration="1.770473713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:35:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:35:03.769184822 +0000 UTC m=+527.635311477" watchObservedRunningTime="2026-07-10 13:35:03.770473713 +0000 UTC m=+527.636600431"
Jul 10 13:35:03.771409 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:03.771379    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-tlmxw\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tlmxw"
Jul 10 13:35:04.664962 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:04.664931    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c92c25-1393-46b5-9cf8-c52a39098880" path="/var/lib/kubelet/pods/27c92c25-1393-46b5-9cf8-c52a39098880/volumes"
Jul 10 13:35:04.665265 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:04.665251    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d" path="/var/lib/kubelet/pods/3daae0c1-e4a0-48d0-bd66-dcfcde0fad2d/volumes"
Jul 10 13:35:04.665546 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:04.665535    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eefa17b-ca65-47a4-a8ce-495e29bb00f1" path="/var/lib/kubelet/pods/5eefa17b-ca65-47a4-a8ce-495e29bb00f1/volumes"
Jul 10 13:35:14.708756 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:14.708677    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5srn4"
Jul 10 13:35:14.709275 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:14.708864    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:35:14.709275 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:14.708925    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:14.807945 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:14.807917    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"]
Jul 10 13:35:14.808096 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:14.808078    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt" podUID="a8789a79-0a3f-4365-9328-7833511eb136" containerName="manager" containerID="cri-o://3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c" gracePeriod=10
Jul 10 13:35:15.044965 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.044940    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:15.162239 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.162216    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8789a79-0a3f-4365-9328-7833511eb136-extensions-socket-volume\") pod \"a8789a79-0a3f-4365-9328-7833511eb136\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") "
Jul 10 13:35:15.162356 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.162264    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqtp8\" (UniqueName: \"kubernetes.io/projected/a8789a79-0a3f-4365-9328-7833511eb136-kube-api-access-hqtp8\") pod \"a8789a79-0a3f-4365-9328-7833511eb136\" (UID: \"a8789a79-0a3f-4365-9328-7833511eb136\") "
Jul 10 13:35:15.162565 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.162543    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8789a79-0a3f-4365-9328-7833511eb136-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a8789a79-0a3f-4365-9328-7833511eb136" (UID: "a8789a79-0a3f-4365-9328-7833511eb136"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:35:15.164061 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.164040    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8789a79-0a3f-4365-9328-7833511eb136-kube-api-access-hqtp8" (OuterVolumeSpecName: "kube-api-access-hqtp8") pod "a8789a79-0a3f-4365-9328-7833511eb136" (UID: "a8789a79-0a3f-4365-9328-7833511eb136"). InnerVolumeSpecName "kube-api-access-hqtp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:35:15.263112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.263063    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqtp8\" (UniqueName: \"kubernetes.io/projected/a8789a79-0a3f-4365-9328-7833511eb136-kube-api-access-hqtp8\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:15.263112 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.263081    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8789a79-0a3f-4365-9328-7833511eb136-extensions-socket-volume\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:15.759755 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.759729    2554 generic.go:356] "Generic (PLEG): container finished" podID="a8789a79-0a3f-4365-9328-7833511eb136" containerID="3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c" exitCode=0
Jul 10 13:35:15.760093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.759775    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"
Jul 10 13:35:15.760093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.759835    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt" event={"ID":"a8789a79-0a3f-4365-9328-7833511eb136","Type":"ContainerDied","Data":"3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c"}
Jul 10 13:35:15.760093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.759887    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt" event={"ID":"a8789a79-0a3f-4365-9328-7833511eb136","Type":"ContainerDied","Data":"9c77d461ed228c127c7042f72aaada9c5009a3457c280cef3eb5b9fc36afb0c0"}
Jul 10 13:35:15.760093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.759914    2554 scope.go:117] "RemoveContainer" containerID="3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c"
Jul 10 13:35:15.768729 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.768710    2554 scope.go:117] "RemoveContainer" containerID="3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c"
Jul 10 13:35:15.769000 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:15.768978    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c\": container with ID starting with 3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c not found: ID does not exist" containerID="3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c"
Jul 10 13:35:15.769058 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.769007    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c"} err="failed to get container status \"3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c\": rpc error: code = NotFound desc = could not find container \"3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c\": container with ID starting with 3fae6b9e8250f81bb17fb647198bb116a2c71d47b401db186cd4f66c07c05d3c not found: ID does not exist"
Jul 10 13:35:15.795207 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.795187    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"]
Jul 10 13:35:15.800901 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:15.800883    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f6smt"]
Jul 10 13:35:16.662808 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:16.662779    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8789a79-0a3f-4365-9328-7833511eb136" path="/var/lib/kubelet/pods/a8789a79-0a3f-4365-9328-7833511eb136/volumes"
Jul 10 13:35:33.067017 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.066984    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vllw"]
Jul 10 13:35:33.067376 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.067275    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8789a79-0a3f-4365-9328-7833511eb136" containerName="manager"
Jul 10 13:35:33.067376 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.067287    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8789a79-0a3f-4365-9328-7833511eb136" containerName="manager"
Jul 10 13:35:33.067376 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.067351    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8789a79-0a3f-4365-9328-7833511eb136" containerName="manager"
Jul 10 13:35:33.072013 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.071994    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.075079 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.075058    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 10 13:35:33.075179 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.075096    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vwzg9\""
Jul 10 13:35:33.080002 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.079983    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vllw"]
Jul 10 13:35:33.161888 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.161859    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vllw"]
Jul 10 13:35:33.191123 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.191100    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-config-file\") pod \"limitador-limitador-64c94fd64f-4vllw\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.191261 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.191139    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db72s\" (UniqueName: \"kubernetes.io/projected/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-kube-api-access-db72s\") pod \"limitador-limitador-64c94fd64f-4vllw\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.291776 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.291749    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-config-file\") pod \"limitador-limitador-64c94fd64f-4vllw\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.291920 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.291787    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db72s\" (UniqueName: \"kubernetes.io/projected/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-kube-api-access-db72s\") pod \"limitador-limitador-64c94fd64f-4vllw\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.292338 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.292321    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-config-file\") pod \"limitador-limitador-64c94fd64f-4vllw\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.301510 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.301484    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db72s\" (UniqueName: \"kubernetes.io/projected/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-kube-api-access-db72s\") pod \"limitador-limitador-64c94fd64f-4vllw\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.383837 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.383767    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:33.521978 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.521950    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vllw"]
Jul 10 13:35:33.524869 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:35:33.524841    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acd1b52_39d3_4f5d_8b57_21bdafebcc6f.slice/crio-a720d5e58548bbbe331e8ed8481d6a25481e34be6d60e424ff1304ccf5437940 WatchSource:0}: Error finding container a720d5e58548bbbe331e8ed8481d6a25481e34be6d60e424ff1304ccf5437940: Status 404 returned error can't find the container with id a720d5e58548bbbe331e8ed8481d6a25481e34be6d60e424ff1304ccf5437940
Jul 10 13:35:33.836579 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.836553    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw" event={"ID":"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f","Type":"ContainerStarted","Data":"a720d5e58548bbbe331e8ed8481d6a25481e34be6d60e424ff1304ccf5437940"}
Jul 10 13:35:33.943663 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.943634    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6lfdm"]
Jul 10 13:35:33.946953 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.946935    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lfdm"
Jul 10 13:35:33.949910 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.949859    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8wsmg\""
Jul 10 13:35:33.954534 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.954488    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lfdm"]
Jul 10 13:35:33.997310 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:33.997286    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smzpd\" (UniqueName: \"kubernetes.io/projected/8eda1441-6855-4896-88e4-161977b4ad0a-kube-api-access-smzpd\") pod \"authorino-58cd955675-6lfdm\" (UID: \"8eda1441-6855-4896-88e4-161977b4ad0a\") " pod="kuadrant-system/authorino-58cd955675-6lfdm"
Jul 10 13:35:34.098025 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.097966    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smzpd\" (UniqueName: \"kubernetes.io/projected/8eda1441-6855-4896-88e4-161977b4ad0a-kube-api-access-smzpd\") pod \"authorino-58cd955675-6lfdm\" (UID: \"8eda1441-6855-4896-88e4-161977b4ad0a\") " pod="kuadrant-system/authorino-58cd955675-6lfdm"
Jul 10 13:35:34.114253 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.114226    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smzpd\" (UniqueName: \"kubernetes.io/projected/8eda1441-6855-4896-88e4-161977b4ad0a-kube-api-access-smzpd\") pod \"authorino-58cd955675-6lfdm\" (UID: \"8eda1441-6855-4896-88e4-161977b4ad0a\") " pod="kuadrant-system/authorino-58cd955675-6lfdm"
Jul 10 13:35:34.161999 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.161974    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-cbw22"]
Jul 10 13:35:34.163944 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.163930    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cbw22"
Jul 10 13:35:34.172646 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.172625    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cbw22"]
Jul 10 13:35:34.257107 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.257078    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lfdm"
Jul 10 13:35:34.300193 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.300143    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6gv\" (UniqueName: \"kubernetes.io/projected/d4ca1631-cca5-4af6-863c-31c3ac5c7c02-kube-api-access-vt6gv\") pod \"authorino-75d556bd77-cbw22\" (UID: \"d4ca1631-cca5-4af6-863c-31c3ac5c7c02\") " pod="kuadrant-system/authorino-75d556bd77-cbw22"
Jul 10 13:35:34.402151 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.401707    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6gv\" (UniqueName: \"kubernetes.io/projected/d4ca1631-cca5-4af6-863c-31c3ac5c7c02-kube-api-access-vt6gv\") pod \"authorino-75d556bd77-cbw22\" (UID: \"d4ca1631-cca5-4af6-863c-31c3ac5c7c02\") " pod="kuadrant-system/authorino-75d556bd77-cbw22"
Jul 10 13:35:34.418945 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.415527    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6gv\" (UniqueName: \"kubernetes.io/projected/d4ca1631-cca5-4af6-863c-31c3ac5c7c02-kube-api-access-vt6gv\") pod \"authorino-75d556bd77-cbw22\" (UID: \"d4ca1631-cca5-4af6-863c-31c3ac5c7c02\") " pod="kuadrant-system/authorino-75d556bd77-cbw22"
Jul 10 13:35:34.423682 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.422764    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lfdm"]
Jul 10 13:35:34.426908 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:35:34.426106    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8eda1441_6855_4896_88e4_161977b4ad0a.slice/crio-bb01065c8d336ffbd8a7283f4d8826d419bb82523bdd981f32c9df66a5510c0b WatchSource:0}: Error finding container bb01065c8d336ffbd8a7283f4d8826d419bb82523bdd981f32c9df66a5510c0b: Status 404 returned error can't find the container with id bb01065c8d336ffbd8a7283f4d8826d419bb82523bdd981f32c9df66a5510c0b
Jul 10 13:35:34.473411 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.473385    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cbw22"
Jul 10 13:35:34.640070 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.640006    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cbw22"]
Jul 10 13:35:34.643030 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:35:34.643000    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4ca1631_cca5_4af6_863c_31c3ac5c7c02.slice/crio-7f9eb8ad81f2eaaa39cb91aa73ab4667fe6c277f2058d7f643fc8650e6ec58c9 WatchSource:0}: Error finding container 7f9eb8ad81f2eaaa39cb91aa73ab4667fe6c277f2058d7f643fc8650e6ec58c9: Status 404 returned error can't find the container with id 7f9eb8ad81f2eaaa39cb91aa73ab4667fe6c277f2058d7f643fc8650e6ec58c9
Jul 10 13:35:34.842431 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.842370    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lfdm" event={"ID":"8eda1441-6855-4896-88e4-161977b4ad0a","Type":"ContainerStarted","Data":"bb01065c8d336ffbd8a7283f4d8826d419bb82523bdd981f32c9df66a5510c0b"}
Jul 10 13:35:34.843700 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:34.843666    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cbw22" event={"ID":"d4ca1631-cca5-4af6-863c-31c3ac5c7c02","Type":"ContainerStarted","Data":"7f9eb8ad81f2eaaa39cb91aa73ab4667fe6c277f2058d7f643fc8650e6ec58c9"}
Jul 10 13:35:38.865671 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.865633    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw" event={"ID":"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f","Type":"ContainerStarted","Data":"7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669"}
Jul 10 13:35:38.866142 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.865810    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:38.867267 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.867242    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lfdm" event={"ID":"8eda1441-6855-4896-88e4-161977b4ad0a","Type":"ContainerStarted","Data":"6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416"}
Jul 10 13:35:38.868605 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.868580    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cbw22" event={"ID":"d4ca1631-cca5-4af6-863c-31c3ac5c7c02","Type":"ContainerStarted","Data":"cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187"}
Jul 10 13:35:38.884217 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.884182    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw" podStartSLOduration=1.2923866990000001 podStartE2EDuration="5.884168847s" totalImagesPullingTime="4.591782148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:35:38.883655469 +0000 UTC m=+562.749782125" watchObservedRunningTime="2026-07-10 13:35:38.884168847 +0000 UTC m=+562.750295501"
Jul 10 13:35:38.900118 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.900079    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6lfdm" podStartSLOduration=2.166158916 podStartE2EDuration="5.900067054s" totalImagesPullingTime="3.733908138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:35:38.898519743 +0000 UTC m=+562.764646397" watchObservedRunningTime="2026-07-10 13:35:38.900067054 +0000 UTC m=+562.766193707"
Jul 10 13:35:38.914649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.914614    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-cbw22" podStartSLOduration=1.402814824 podStartE2EDuration="4.914600423s" totalImagesPullingTime="3.511785599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:35:38.91258664 +0000 UTC m=+562.778713295" watchObservedRunningTime="2026-07-10 13:35:38.914600423 +0000 UTC m=+562.780727078"
Jul 10 13:35:38.951841 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:38.951797    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lfdm"]
Jul 10 13:35:40.877049 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:40.877011    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6lfdm" podUID="8eda1441-6855-4896-88e4-161977b4ad0a" containerName="authorino" containerID="cri-o://6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416" gracePeriod=30
Jul 10 13:35:41.126751 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.126728    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lfdm"
Jul 10 13:35:41.261468 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.261437    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smzpd\" (UniqueName: \"kubernetes.io/projected/8eda1441-6855-4896-88e4-161977b4ad0a-kube-api-access-smzpd\") pod \"8eda1441-6855-4896-88e4-161977b4ad0a\" (UID: \"8eda1441-6855-4896-88e4-161977b4ad0a\") "
Jul 10 13:35:41.263389 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.263366    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eda1441-6855-4896-88e4-161977b4ad0a-kube-api-access-smzpd" (OuterVolumeSpecName: "kube-api-access-smzpd") pod "8eda1441-6855-4896-88e4-161977b4ad0a" (UID: "8eda1441-6855-4896-88e4-161977b4ad0a"). InnerVolumeSpecName "kube-api-access-smzpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:35:41.362830 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.362803    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smzpd\" (UniqueName: \"kubernetes.io/projected/8eda1441-6855-4896-88e4-161977b4ad0a-kube-api-access-smzpd\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:41.882516 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.882485    2554 generic.go:356] "Generic (PLEG): container finished" podID="8eda1441-6855-4896-88e4-161977b4ad0a" containerID="6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416" exitCode=0
Jul 10 13:35:41.882926 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.882523    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6lfdm"
Jul 10 13:35:41.882926 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.882561    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lfdm" event={"ID":"8eda1441-6855-4896-88e4-161977b4ad0a","Type":"ContainerDied","Data":"6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416"}
Jul 10 13:35:41.882926 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.882589    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6lfdm" event={"ID":"8eda1441-6855-4896-88e4-161977b4ad0a","Type":"ContainerDied","Data":"bb01065c8d336ffbd8a7283f4d8826d419bb82523bdd981f32c9df66a5510c0b"}
Jul 10 13:35:41.882926 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.882605    2554 scope.go:117] "RemoveContainer" containerID="6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416"
Jul 10 13:35:41.891306 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.891291    2554 scope.go:117] "RemoveContainer" containerID="6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416"
Jul 10 13:35:41.891528 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:41.891510    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416\": container with ID starting with 6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416 not found: ID does not exist" containerID="6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416"
Jul 10 13:35:41.891585 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.891536    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416"} err="failed to get container status \"6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416\": rpc error: code = NotFound desc = could not find container \"6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416\": container with ID starting with 6721ef961356819c03a4ba44d2975badedb7afed5158996837c6cc6f200b1416 not found: ID does not exist"
Jul 10 13:35:41.906128 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.906108    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lfdm"]
Jul 10 13:35:41.911561 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:41.911540    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6lfdm"]
Jul 10 13:35:42.663426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:42.663395    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eda1441-6855-4896-88e4-161977b4ad0a" path="/var/lib/kubelet/pods/8eda1441-6855-4896-88e4-161977b4ad0a/volumes"
Jul 10 13:35:48.576590 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:48.576559    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vllw"]
Jul 10 13:35:48.577140 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:48.576747    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw" podUID="0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" containerName="limitador" containerID="cri-o://7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669" gracePeriod=30
Jul 10 13:35:48.577542 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:48.577415    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:49.117654 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.117635    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:49.215569 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.215543    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-config-file\") pod \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") "
Jul 10 13:35:49.215707 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.215581    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db72s\" (UniqueName: \"kubernetes.io/projected/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-kube-api-access-db72s\") pod \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\" (UID: \"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f\") "
Jul 10 13:35:49.215945 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.215922    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-config-file" (OuterVolumeSpecName: "config-file") pod "0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" (UID: "0acd1b52-39d3-4f5d-8b57-21bdafebcc6f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:35:49.217530 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.217505    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-kube-api-access-db72s" (OuterVolumeSpecName: "kube-api-access-db72s") pod "0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" (UID: "0acd1b52-39d3-4f5d-8b57-21bdafebcc6f"). InnerVolumeSpecName "kube-api-access-db72s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:35:49.316937 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.316912    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db72s\" (UniqueName: \"kubernetes.io/projected/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-kube-api-access-db72s\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:49.316937 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.316936    2554 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f-config-file\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:35:49.917951 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.917923    2554 generic.go:356] "Generic (PLEG): container finished" podID="0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" containerID="7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669" exitCode=0
Jul 10 13:35:49.918382 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.917974    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw"
Jul 10 13:35:49.918382 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.918014    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw" event={"ID":"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f","Type":"ContainerDied","Data":"7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669"}
Jul 10 13:35:49.918382 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.918068    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4vllw" event={"ID":"0acd1b52-39d3-4f5d-8b57-21bdafebcc6f","Type":"ContainerDied","Data":"a720d5e58548bbbe331e8ed8481d6a25481e34be6d60e424ff1304ccf5437940"}
Jul 10 13:35:49.918382 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.918090    2554 scope.go:117] "RemoveContainer" containerID="7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669"
Jul 10 13:35:49.926689 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.926672    2554 scope.go:117] "RemoveContainer" containerID="7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669"
Jul 10 13:35:49.926984 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:35:49.926962    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669\": container with ID starting with 7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669 not found: ID does not exist" containerID="7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669"
Jul 10 13:35:49.927050 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.926993    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669"} err="failed to get container status \"7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669\": rpc error: code = NotFound desc = could not find container \"7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669\": container with ID starting with 7c05f9fdb920369a22a13c314cbeaabe14577f3dc98dbb1194d4b028922b6669 not found: ID does not exist"
Jul 10 13:35:49.942060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.942031    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vllw"]
Jul 10 13:35:49.949209 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:49.949189    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4vllw"]
Jul 10 13:35:50.663535 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:50.663506    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" path="/var/lib/kubelet/pods/0acd1b52-39d3-4f5d-8b57-21bdafebcc6f/volumes"
Jul 10 13:35:54.903307 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.903272    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"]
Jul 10 13:35:54.903766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.903745    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" containerName="limitador"
Jul 10 13:35:54.903856 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.903770    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" containerName="limitador"
Jul 10 13:35:54.903856 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.903785    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eda1441-6855-4896-88e4-161977b4ad0a" containerName="authorino"
Jul 10 13:35:54.903856 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.903793    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eda1441-6855-4896-88e4-161977b4ad0a" containerName="authorino"
Jul 10 13:35:54.904006 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.903900    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eda1441-6855-4896-88e4-161977b4ad0a" containerName="authorino"
Jul 10 13:35:54.904006 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.903917    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0acd1b52-39d3-4f5d-8b57-21bdafebcc6f" containerName="limitador"
Jul 10 13:35:54.906925 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.906902    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:54.909684 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.909661    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 13:35:54.909852 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.909701    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 10 13:35:54.910794 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.910773    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-h89sl\""
Jul 10 13:35:54.910922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.910794    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 13:35:54.917773 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:54.917753    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"]
Jul 10 13:35:55.055086 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.055064    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnnjj\" (UniqueName: \"kubernetes.io/projected/f3d51f2c-0656-4c1c-a63a-0cbcafb335f6-kube-api-access-rnnjj\") pod \"postgres-69fb4594cf-p44q4\" (UID: \"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:55.055176 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.055111    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3d51f2c-0656-4c1c-a63a-0cbcafb335f6-data\") pod \"postgres-69fb4594cf-p44q4\" (UID: \"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:55.155930 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.155880    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnnjj\" (UniqueName: \"kubernetes.io/projected/f3d51f2c-0656-4c1c-a63a-0cbcafb335f6-kube-api-access-rnnjj\") pod \"postgres-69fb4594cf-p44q4\" (UID: \"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:55.155930 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.155924    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3d51f2c-0656-4c1c-a63a-0cbcafb335f6-data\") pod \"postgres-69fb4594cf-p44q4\" (UID: \"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:55.156235 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.156221    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f3d51f2c-0656-4c1c-a63a-0cbcafb335f6-data\") pod \"postgres-69fb4594cf-p44q4\" (UID: \"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:55.165865 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.165811    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnnjj\" (UniqueName: \"kubernetes.io/projected/f3d51f2c-0656-4c1c-a63a-0cbcafb335f6-kube-api-access-rnnjj\") pod \"postgres-69fb4594cf-p44q4\" (UID: \"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:55.219501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.219477    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:55.349805 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.349738    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"]
Jul 10 13:35:55.351961 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:35:55.351933    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d51f2c_0656_4c1c_a63a_0cbcafb335f6.slice/crio-7f42c38bd1f2a42b7e9ee220fcfebc8d0e6fd459ee2a849782404d9cfed1373f WatchSource:0}: Error finding container 7f42c38bd1f2a42b7e9ee220fcfebc8d0e6fd459ee2a849782404d9cfed1373f: Status 404 returned error can't find the container with id 7f42c38bd1f2a42b7e9ee220fcfebc8d0e6fd459ee2a849782404d9cfed1373f
Jul 10 13:35:55.949364 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:55.949277    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4" event={"ID":"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6","Type":"ContainerStarted","Data":"7f42c38bd1f2a42b7e9ee220fcfebc8d0e6fd459ee2a849782404d9cfed1373f"}
Jul 10 13:35:59.971199 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:59.971163    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4" event={"ID":"f3d51f2c-0656-4c1c-a63a-0cbcafb335f6","Type":"ContainerStarted","Data":"ca3199b32586f8b003aafdd1314dc51c7c5557cfd2b0a4ea262834244bdc9bfd"}
Jul 10 13:35:59.971617 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:59.971267    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:35:59.991021 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:35:59.990977    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4" podStartSLOduration=1.484153925 podStartE2EDuration="5.990961404s" totalImagesPullingTime="4.506807479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:35:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:35:59.988464552 +0000 UTC m=+583.854591207" watchObservedRunningTime="2026-07-10 13:35:59.990961404 +0000 UTC m=+583.857088089"
Jul 10 13:36:06.004365 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.004338    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p44q4"
Jul 10 13:36:06.600647 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.600613    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgtg9"]
Jul 10 13:36:06.641001 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.640970    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgtg9"]
Jul 10 13:36:06.641131 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.641067    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgtg9"
Jul 10 13:36:06.746006 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.745974    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nslr\" (UniqueName: \"kubernetes.io/projected/73e51d45-046a-441e-93a0-eabe0454bc86-kube-api-access-4nslr\") pod \"authorino-5d4b67c446-tgtg9\" (UID: \"73e51d45-046a-441e-93a0-eabe0454bc86\") " pod="kuadrant-system/authorino-5d4b67c446-tgtg9"
Jul 10 13:36:06.846716 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.846690    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nslr\" (UniqueName: \"kubernetes.io/projected/73e51d45-046a-441e-93a0-eabe0454bc86-kube-api-access-4nslr\") pod \"authorino-5d4b67c446-tgtg9\" (UID: \"73e51d45-046a-441e-93a0-eabe0454bc86\") " pod="kuadrant-system/authorino-5d4b67c446-tgtg9"
Jul 10 13:36:06.848160 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.848139    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgtg9"]
Jul 10 13:36:06.848315 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:36:06.848298    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4nslr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-tgtg9" podUID="73e51d45-046a-441e-93a0-eabe0454bc86"
Jul 10 13:36:06.857326 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.857273    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nslr\" (UniqueName: \"kubernetes.io/projected/73e51d45-046a-441e-93a0-eabe0454bc86-kube-api-access-4nslr\") pod \"authorino-5d4b67c446-tgtg9\" (UID: \"73e51d45-046a-441e-93a0-eabe0454bc86\") " pod="kuadrant-system/authorino-5d4b67c446-tgtg9"
Jul 10 13:36:06.873231 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.873208    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f8cf66858-9nd5p"]
Jul 10 13:36:06.895855 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.895836    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f8cf66858-9nd5p"]
Jul 10 13:36:06.895941 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.895920    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8cf66858-9nd5p"
Jul 10 13:36:06.968508 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:06.968483    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f8cf66858-9nd5p"]
Jul 10 13:36:06.968659 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:36:06.968643    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zs42x], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7f8cf66858-9nd5p" podUID="d25a7054-139e-4d63-bb55-2d162bb4a9c2"
Jul 10 13:36:07.000863 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.000834    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7fdcb44678-mn6qx"]
Jul 10 13:36:07.002095 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.002073    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8cf66858-9nd5p"
Jul 10 13:36:07.002201 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.002080    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgtg9"
Jul 10 13:36:07.017766 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.017746    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.018128 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.017748    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdcb44678-mn6qx"]
Jul 10 13:36:07.019342 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.019326    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8cf66858-9nd5p"
Jul 10 13:36:07.020195 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.020162    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 13:36:07.024304 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.024283    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgtg9"
Jul 10 13:36:07.048847 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.048809    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs42x\" (UniqueName: \"kubernetes.io/projected/d25a7054-139e-4d63-bb55-2d162bb4a9c2-kube-api-access-zs42x\") pod \"authorino-7f8cf66858-9nd5p\" (UID: \"d25a7054-139e-4d63-bb55-2d162bb4a9c2\") " pod="kuadrant-system/authorino-7f8cf66858-9nd5p"
Jul 10 13:36:07.149425 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.149378    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nslr\" (UniqueName: \"kubernetes.io/projected/73e51d45-046a-441e-93a0-eabe0454bc86-kube-api-access-4nslr\") pod \"73e51d45-046a-441e-93a0-eabe0454bc86\" (UID: \"73e51d45-046a-441e-93a0-eabe0454bc86\") "
Jul 10 13:36:07.149502 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.149490    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk99v\" (UniqueName: \"kubernetes.io/projected/426d50a5-4e40-4e1a-b829-8df81808f086-kube-api-access-zk99v\") pod \"authorino-7fdcb44678-mn6qx\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") " pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.149545 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.149526    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs42x\" (UniqueName: \"kubernetes.io/projected/d25a7054-139e-4d63-bb55-2d162bb4a9c2-kube-api-access-zs42x\") pod \"authorino-7f8cf66858-9nd5p\" (UID: \"d25a7054-139e-4d63-bb55-2d162bb4a9c2\") " pod="kuadrant-system/authorino-7f8cf66858-9nd5p"
Jul 10 13:36:07.149590 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.149550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/426d50a5-4e40-4e1a-b829-8df81808f086-tls-cert\") pod \"authorino-7fdcb44678-mn6qx\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") " pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.151375 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.151342    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73e51d45-046a-441e-93a0-eabe0454bc86-kube-api-access-4nslr" (OuterVolumeSpecName: "kube-api-access-4nslr") pod "73e51d45-046a-441e-93a0-eabe0454bc86" (UID: "73e51d45-046a-441e-93a0-eabe0454bc86"). InnerVolumeSpecName "kube-api-access-4nslr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:36:07.159372 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.159342    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs42x\" (UniqueName: \"kubernetes.io/projected/d25a7054-139e-4d63-bb55-2d162bb4a9c2-kube-api-access-zs42x\") pod \"authorino-7f8cf66858-9nd5p\" (UID: \"d25a7054-139e-4d63-bb55-2d162bb4a9c2\") " pod="kuadrant-system/authorino-7f8cf66858-9nd5p"
Jul 10 13:36:07.250377 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.250359    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs42x\" (UniqueName: \"kubernetes.io/projected/d25a7054-139e-4d63-bb55-2d162bb4a9c2-kube-api-access-zs42x\") pod \"d25a7054-139e-4d63-bb55-2d162bb4a9c2\" (UID: \"d25a7054-139e-4d63-bb55-2d162bb4a9c2\") "
Jul 10 13:36:07.250485 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.250472    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/426d50a5-4e40-4e1a-b829-8df81808f086-tls-cert\") pod \"authorino-7fdcb44678-mn6qx\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") " pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.250534 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.250511    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk99v\" (UniqueName: \"kubernetes.io/projected/426d50a5-4e40-4e1a-b829-8df81808f086-kube-api-access-zk99v\") pod \"authorino-7fdcb44678-mn6qx\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") " pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.250570 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.250544    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nslr\" (UniqueName: \"kubernetes.io/projected/73e51d45-046a-441e-93a0-eabe0454bc86-kube-api-access-4nslr\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:36:07.252160 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.252134    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d25a7054-139e-4d63-bb55-2d162bb4a9c2-kube-api-access-zs42x" (OuterVolumeSpecName: "kube-api-access-zs42x") pod "d25a7054-139e-4d63-bb55-2d162bb4a9c2" (UID: "d25a7054-139e-4d63-bb55-2d162bb4a9c2"). InnerVolumeSpecName "kube-api-access-zs42x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:36:07.252692 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.252676    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/426d50a5-4e40-4e1a-b829-8df81808f086-tls-cert\") pod \"authorino-7fdcb44678-mn6qx\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") " pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.259428 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.259408    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk99v\" (UniqueName: \"kubernetes.io/projected/426d50a5-4e40-4e1a-b829-8df81808f086-kube-api-access-zk99v\") pod \"authorino-7fdcb44678-mn6qx\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") " pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.326342 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.326323    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:36:07.351145 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.351124    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs42x\" (UniqueName: \"kubernetes.io/projected/d25a7054-139e-4d63-bb55-2d162bb4a9c2-kube-api-access-zs42x\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:36:07.477238 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:07.477215    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdcb44678-mn6qx"]
Jul 10 13:36:07.479700 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:36:07.479671    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod426d50a5_4e40_4e1a_b829_8df81808f086.slice/crio-64a5b73fc5a83059cf4cf84555aac41446655a385ab0c4b0ef115580ae7b633f WatchSource:0}: Error finding container 64a5b73fc5a83059cf4cf84555aac41446655a385ab0c4b0ef115580ae7b633f: Status 404 returned error can't find the container with id 64a5b73fc5a83059cf4cf84555aac41446655a385ab0c4b0ef115580ae7b633f
Jul 10 13:36:08.008765 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.008727    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdcb44678-mn6qx" event={"ID":"426d50a5-4e40-4e1a-b829-8df81808f086","Type":"ContainerStarted","Data":"35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa"}
Jul 10 13:36:08.008944 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.008777    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdcb44678-mn6qx" event={"ID":"426d50a5-4e40-4e1a-b829-8df81808f086","Type":"ContainerStarted","Data":"64a5b73fc5a83059cf4cf84555aac41446655a385ab0c4b0ef115580ae7b633f"}
Jul 10 13:36:08.008944 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.008913    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f8cf66858-9nd5p"
Jul 10 13:36:08.008944 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.008934    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgtg9"
Jul 10 13:36:08.029569 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.029523    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7fdcb44678-mn6qx" podStartSLOduration=1.611986125 podStartE2EDuration="2.029509183s" totalImagesPullingTime="417.523058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:36:08.028046732 +0000 UTC m=+591.894173387" watchObservedRunningTime="2026-07-10 13:36:08.029509183 +0000 UTC m=+591.895635837"
Jul 10 13:36:08.055441 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.055417    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgtg9"]
Jul 10 13:36:08.061973 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.061834    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cbw22"]
Jul 10 13:36:08.062103 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.062066    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-cbw22" podUID="d4ca1631-cca5-4af6-863c-31c3ac5c7c02" containerName="authorino" containerID="cri-o://cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187" gracePeriod=30
Jul 10 13:36:08.064731 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.064706    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgtg9"]
Jul 10 13:36:08.090145 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.090118    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f8cf66858-9nd5p"]
Jul 10 13:36:08.091750 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:36:08.091728    2554 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4ca1631_cca5_4af6_863c_31c3ac5c7c02.slice/crio-cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187.scope\": RecentStats: unable to find data in memory cache]"
Jul 10 13:36:08.096266 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.096245    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7f8cf66858-9nd5p"]
Jul 10 13:36:08.308006 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.307984    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cbw22"
Jul 10 13:36:08.459285 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.459211    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt6gv\" (UniqueName: \"kubernetes.io/projected/d4ca1631-cca5-4af6-863c-31c3ac5c7c02-kube-api-access-vt6gv\") pod \"d4ca1631-cca5-4af6-863c-31c3ac5c7c02\" (UID: \"d4ca1631-cca5-4af6-863c-31c3ac5c7c02\") "
Jul 10 13:36:08.461262 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.461233    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ca1631-cca5-4af6-863c-31c3ac5c7c02-kube-api-access-vt6gv" (OuterVolumeSpecName: "kube-api-access-vt6gv") pod "d4ca1631-cca5-4af6-863c-31c3ac5c7c02" (UID: "d4ca1631-cca5-4af6-863c-31c3ac5c7c02"). InnerVolumeSpecName "kube-api-access-vt6gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:36:08.560499 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.560475    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt6gv\" (UniqueName: \"kubernetes.io/projected/d4ca1631-cca5-4af6-863c-31c3ac5c7c02-kube-api-access-vt6gv\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:36:08.668889 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.668864    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73e51d45-046a-441e-93a0-eabe0454bc86" path="/var/lib/kubelet/pods/73e51d45-046a-441e-93a0-eabe0454bc86/volumes"
Jul 10 13:36:08.669080 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:08.669068    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d25a7054-139e-4d63-bb55-2d162bb4a9c2" path="/var/lib/kubelet/pods/d25a7054-139e-4d63-bb55-2d162bb4a9c2/volumes"
Jul 10 13:36:09.013606 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.013579    2554 generic.go:356] "Generic (PLEG): container finished" podID="d4ca1631-cca5-4af6-863c-31c3ac5c7c02" containerID="cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187" exitCode=0
Jul 10 13:36:09.013705 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.013631    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cbw22"
Jul 10 13:36:09.013705 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.013665    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cbw22" event={"ID":"d4ca1631-cca5-4af6-863c-31c3ac5c7c02","Type":"ContainerDied","Data":"cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187"}
Jul 10 13:36:09.013705 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.013703    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cbw22" event={"ID":"d4ca1631-cca5-4af6-863c-31c3ac5c7c02","Type":"ContainerDied","Data":"7f9eb8ad81f2eaaa39cb91aa73ab4667fe6c277f2058d7f643fc8650e6ec58c9"}
Jul 10 13:36:09.013807 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.013720    2554 scope.go:117] "RemoveContainer" containerID="cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187"
Jul 10 13:36:09.021739 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.021722    2554 scope.go:117] "RemoveContainer" containerID="cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187"
Jul 10 13:36:09.022019 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:36:09.021998    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187\": container with ID starting with cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187 not found: ID does not exist" containerID="cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187"
Jul 10 13:36:09.022105 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.022024    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187"} err="failed to get container status \"cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187\": rpc error: code = NotFound desc = could not find container \"cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187\": container with ID starting with cc1013f90dd40adcf4041503fffcfae14e5ff99cddc0855508c636ba2b546187 not found: ID does not exist"
Jul 10 13:36:09.033401 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.033379    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cbw22"]
Jul 10 13:36:09.039581 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:09.039559    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cbw22"]
Jul 10 13:36:10.662735 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:10.662705    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ca1631-cca5-4af6-863c-31c3ac5c7c02" path="/var/lib/kubelet/pods/d4ca1631-cca5-4af6-863c-31c3ac5c7c02/volumes"
Jul 10 13:36:12.727026 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.726986    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d99d857bf-kdp9k"]
Jul 10 13:36:12.727494 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.727479    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ca1631-cca5-4af6-863c-31c3ac5c7c02" containerName="authorino"
Jul 10 13:36:12.727550 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.727498    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ca1631-cca5-4af6-863c-31c3ac5c7c02" containerName="authorino"
Jul 10 13:36:12.727589 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.727583    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ca1631-cca5-4af6-863c-31c3ac5c7c02" containerName="authorino"
Jul 10 13:36:12.738320 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.738296    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:12.741515 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.741289    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 13:36:12.741515 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.741413    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 13:36:12.741515 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.741412    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7pq9t\""
Jul 10 13:36:12.742348 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.742322    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d99d857bf-kdp9k"]
Jul 10 13:36:12.888367 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.888339    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/644629cd-4b00-4415-a4b3-a6fef85a2af3-webhook-cert\") pod \"maas-controller-7d99d857bf-kdp9k\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") " pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:12.888531 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.888381    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h4gf\" (UniqueName: \"kubernetes.io/projected/644629cd-4b00-4415-a4b3-a6fef85a2af3-kube-api-access-9h4gf\") pod \"maas-controller-7d99d857bf-kdp9k\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") " pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:12.908388 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.908361    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-v5h7v"]
Jul 10 13:36:12.918349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.918327    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:12.923484 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.923462    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-v5h7v"]
Jul 10 13:36:12.989246 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.989193    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h4gf\" (UniqueName: \"kubernetes.io/projected/644629cd-4b00-4415-a4b3-a6fef85a2af3-kube-api-access-9h4gf\") pod \"maas-controller-7d99d857bf-kdp9k\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") " pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:12.989370 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.989288    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/644629cd-4b00-4415-a4b3-a6fef85a2af3-webhook-cert\") pod \"maas-controller-7d99d857bf-kdp9k\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") " pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:12.991610 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.991588    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/644629cd-4b00-4415-a4b3-a6fef85a2af3-webhook-cert\") pod \"maas-controller-7d99d857bf-kdp9k\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") " pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:12.999994 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:12.999975    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h4gf\" (UniqueName: \"kubernetes.io/projected/644629cd-4b00-4415-a4b3-a6fef85a2af3-kube-api-access-9h4gf\") pod \"maas-controller-7d99d857bf-kdp9k\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") " pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:13.051362 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.051329    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:13.089799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.089773    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/996a92fc-c1a1-4b40-a747-903c753040c5-webhook-cert\") pod \"maas-controller-5999cf9bb8-v5h7v\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") " pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:13.089933 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.089814    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndfbd\" (UniqueName: \"kubernetes.io/projected/996a92fc-c1a1-4b40-a747-903c753040c5-kube-api-access-ndfbd\") pod \"maas-controller-5999cf9bb8-v5h7v\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") " pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:13.186067 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.186042    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d99d857bf-kdp9k"]
Jul 10 13:36:13.188128 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:36:13.188105    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644629cd_4b00_4415_a4b3_a6fef85a2af3.slice/crio-36104fe68fcce8a771e0c3790937d2674df2a1933c694e1c63d8e358054c4e0e WatchSource:0}: Error finding container 36104fe68fcce8a771e0c3790937d2674df2a1933c694e1c63d8e358054c4e0e: Status 404 returned error can't find the container with id 36104fe68fcce8a771e0c3790937d2674df2a1933c694e1c63d8e358054c4e0e
Jul 10 13:36:13.190160 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.190135    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/996a92fc-c1a1-4b40-a747-903c753040c5-webhook-cert\") pod \"maas-controller-5999cf9bb8-v5h7v\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") " pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:13.190264 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.190189    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndfbd\" (UniqueName: \"kubernetes.io/projected/996a92fc-c1a1-4b40-a747-903c753040c5-kube-api-access-ndfbd\") pod \"maas-controller-5999cf9bb8-v5h7v\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") " pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:13.192507 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.192486    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/996a92fc-c1a1-4b40-a747-903c753040c5-webhook-cert\") pod \"maas-controller-5999cf9bb8-v5h7v\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") " pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:13.199677 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.199656    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndfbd\" (UniqueName: \"kubernetes.io/projected/996a92fc-c1a1-4b40-a747-903c753040c5-kube-api-access-ndfbd\") pod \"maas-controller-5999cf9bb8-v5h7v\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") " pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:13.229014 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.228990    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:13.377540 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:13.377511    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-v5h7v"]
Jul 10 13:36:13.378969 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:36:13.378943    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996a92fc_c1a1_4b40_a747_903c753040c5.slice/crio-de7db6f8ffe80db3ff764fc5d1669bdfb480ee734da7f0b89b616f50d5a429be WatchSource:0}: Error finding container de7db6f8ffe80db3ff764fc5d1669bdfb480ee734da7f0b89b616f50d5a429be: Status 404 returned error can't find the container with id de7db6f8ffe80db3ff764fc5d1669bdfb480ee734da7f0b89b616f50d5a429be
Jul 10 13:36:14.040499 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:14.040464    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v" event={"ID":"996a92fc-c1a1-4b40-a747-903c753040c5","Type":"ContainerStarted","Data":"de7db6f8ffe80db3ff764fc5d1669bdfb480ee734da7f0b89b616f50d5a429be"}
Jul 10 13:36:14.042154 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:14.042125    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d99d857bf-kdp9k" event={"ID":"644629cd-4b00-4415-a4b3-a6fef85a2af3","Type":"ContainerStarted","Data":"36104fe68fcce8a771e0c3790937d2674df2a1933c694e1c63d8e358054c4e0e"}
Jul 10 13:36:16.052669 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:16.052633    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v" event={"ID":"996a92fc-c1a1-4b40-a747-903c753040c5","Type":"ContainerStarted","Data":"c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604"}
Jul 10 13:36:16.053119 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:16.052814    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:16.054329 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:16.054305    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d99d857bf-kdp9k" event={"ID":"644629cd-4b00-4415-a4b3-a6fef85a2af3","Type":"ContainerStarted","Data":"1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd"}
Jul 10 13:36:16.054434 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:16.054359    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:16.072809 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:16.072767    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v" podStartSLOduration=1.520954025 podStartE2EDuration="4.072750819s" totalImagesPullingTime="2.551796794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:36:16.071906579 +0000 UTC m=+599.938033235" watchObservedRunningTime="2026-07-10 13:36:16.072750819 +0000 UTC m=+599.938877479"
Jul 10 13:36:16.092481 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:16.092439    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d99d857bf-kdp9k" podStartSLOduration=1.358670946 podStartE2EDuration="4.092425954s" totalImagesPullingTime="2.733755008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:36:16.091077298 +0000 UTC m=+599.957203954" watchObservedRunningTime="2026-07-10 13:36:16.092425954 +0000 UTC m=+599.958552607"
Jul 10 13:36:27.065534 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.065499    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:36:27.066151 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.065764    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:27.140060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.140029    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d99d857bf-kdp9k"]
Jul 10 13:36:27.140234 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.140213    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d99d857bf-kdp9k" podUID="644629cd-4b00-4415-a4b3-a6fef85a2af3" containerName="manager" containerID="cri-o://1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd" gracePeriod=30
Jul 10 13:36:27.378138 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.378116    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:27.405660 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.405636    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/644629cd-4b00-4415-a4b3-a6fef85a2af3-webhook-cert\") pod \"644629cd-4b00-4415-a4b3-a6fef85a2af3\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") "
Jul 10 13:36:27.405797 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.405683    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h4gf\" (UniqueName: \"kubernetes.io/projected/644629cd-4b00-4415-a4b3-a6fef85a2af3-kube-api-access-9h4gf\") pod \"644629cd-4b00-4415-a4b3-a6fef85a2af3\" (UID: \"644629cd-4b00-4415-a4b3-a6fef85a2af3\") "
Jul 10 13:36:27.407870 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.407838    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/644629cd-4b00-4415-a4b3-a6fef85a2af3-kube-api-access-9h4gf" (OuterVolumeSpecName: "kube-api-access-9h4gf") pod "644629cd-4b00-4415-a4b3-a6fef85a2af3" (UID: "644629cd-4b00-4415-a4b3-a6fef85a2af3"). InnerVolumeSpecName "kube-api-access-9h4gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:36:27.407955 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.407865    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/644629cd-4b00-4415-a4b3-a6fef85a2af3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "644629cd-4b00-4415-a4b3-a6fef85a2af3" (UID: "644629cd-4b00-4415-a4b3-a6fef85a2af3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:36:27.507096 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.507072    2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/644629cd-4b00-4415-a4b3-a6fef85a2af3-webhook-cert\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:36:27.507096 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:27.507093    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h4gf\" (UniqueName: \"kubernetes.io/projected/644629cd-4b00-4415-a4b3-a6fef85a2af3-kube-api-access-9h4gf\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:36:28.106513 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.106480    2554 generic.go:356] "Generic (PLEG): container finished" podID="644629cd-4b00-4415-a4b3-a6fef85a2af3" containerID="1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd" exitCode=0
Jul 10 13:36:28.106924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.106528    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d99d857bf-kdp9k"
Jul 10 13:36:28.106924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.106560    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d99d857bf-kdp9k" event={"ID":"644629cd-4b00-4415-a4b3-a6fef85a2af3","Type":"ContainerDied","Data":"1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd"}
Jul 10 13:36:28.106924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.106613    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d99d857bf-kdp9k" event={"ID":"644629cd-4b00-4415-a4b3-a6fef85a2af3","Type":"ContainerDied","Data":"36104fe68fcce8a771e0c3790937d2674df2a1933c694e1c63d8e358054c4e0e"}
Jul 10 13:36:28.106924 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.106636    2554 scope.go:117] "RemoveContainer" containerID="1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd"
Jul 10 13:36:28.115631 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.115614    2554 scope.go:117] "RemoveContainer" containerID="1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd"
Jul 10 13:36:28.115930 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:36:28.115911    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd\": container with ID starting with 1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd not found: ID does not exist" containerID="1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd"
Jul 10 13:36:28.115992 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.115938    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd"} err="failed to get container status \"1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd\": rpc error: code = NotFound desc = could not find container \"1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd\": container with ID starting with 1e07f37532bd89936eecb0868af65a031713cbc82965e6e5046dd7b5e5d1e5fd not found: ID does not exist"
Jul 10 13:36:28.131131 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.131108    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d99d857bf-kdp9k"]
Jul 10 13:36:28.136685 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.136664    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d99d857bf-kdp9k"]
Jul 10 13:36:28.664415 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:28.664390    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="644629cd-4b00-4415-a4b3-a6fef85a2af3" path="/var/lib/kubelet/pods/644629cd-4b00-4415-a4b3-a6fef85a2af3/volumes"
Jul 10 13:36:46.898391 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.898311    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84445c5967-8d52g"]
Jul 10 13:36:46.898749 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.898678    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="644629cd-4b00-4415-a4b3-a6fef85a2af3" containerName="manager"
Jul 10 13:36:46.898749 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.898691    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="644629cd-4b00-4415-a4b3-a6fef85a2af3" containerName="manager"
Jul 10 13:36:46.898852 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.898764    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="644629cd-4b00-4415-a4b3-a6fef85a2af3" containerName="manager"
Jul 10 13:36:46.951311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.951267    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84445c5967-8d52g"]
Jul 10 13:36:46.951471 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.951397    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:46.954219 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.954196    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 10 13:36:46.954457 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:46.954439    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-n5j6h\""
Jul 10 13:36:47.063545 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.063509    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqwbc\" (UniqueName: \"kubernetes.io/projected/25af9b04-6c44-4828-a544-a4a06e5340c3-kube-api-access-vqwbc\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.063741 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.063590    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25af9b04-6c44-4828-a544-a4a06e5340c3-openshift-service-ca\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.063741 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.063633    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25af9b04-6c44-4828-a544-a4a06e5340c3-maas-api-tls\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.165013 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.164928    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25af9b04-6c44-4828-a544-a4a06e5340c3-openshift-service-ca\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.165013 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.164970    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25af9b04-6c44-4828-a544-a4a06e5340c3-maas-api-tls\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.165243 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.165034    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqwbc\" (UniqueName: \"kubernetes.io/projected/25af9b04-6c44-4828-a544-a4a06e5340c3-kube-api-access-vqwbc\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.165537 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.165515    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25af9b04-6c44-4828-a544-a4a06e5340c3-openshift-service-ca\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.167296 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.167275    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25af9b04-6c44-4828-a544-a4a06e5340c3-maas-api-tls\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.175203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.175184    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqwbc\" (UniqueName: \"kubernetes.io/projected/25af9b04-6c44-4828-a544-a4a06e5340c3-kube-api-access-vqwbc\") pod \"maas-api-84445c5967-8d52g\" (UID: \"25af9b04-6c44-4828-a544-a4a06e5340c3\") " pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.261991 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.261960    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:47.398339 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.398310    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84445c5967-8d52g"]
Jul 10 13:36:47.400393 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:36:47.400369    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25af9b04_6c44_4828_a544_a4a06e5340c3.slice/crio-a3ca07eb8ef5eb6b56168b42a78a7f2e75e647f54e2b86c1d4f94e8c87c35c50 WatchSource:0}: Error finding container a3ca07eb8ef5eb6b56168b42a78a7f2e75e647f54e2b86c1d4f94e8c87c35c50: Status 404 returned error can't find the container with id a3ca07eb8ef5eb6b56168b42a78a7f2e75e647f54e2b86c1d4f94e8c87c35c50
Jul 10 13:36:47.628646 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.628614    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-g7kzk"]
Jul 10 13:36:47.631429 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.631405    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:47.633957 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.633937    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 10 13:36:47.634045 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.633937    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wprdk\""
Jul 10 13:36:47.641786 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.641767    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-g7kzk"]
Jul 10 13:36:47.769427 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.769378    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf6e7f19-79ab-456b-9552-95545fe4a133-plugins-config-volume\") pod \"payload-pre-processing-7964457696-g7kzk\" (UID: \"bf6e7f19-79ab-456b-9552-95545fe4a133\") " pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:47.769598 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.769438    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrqxq\" (UniqueName: \"kubernetes.io/projected/bf6e7f19-79ab-456b-9552-95545fe4a133-kube-api-access-vrqxq\") pod \"payload-pre-processing-7964457696-g7kzk\" (UID: \"bf6e7f19-79ab-456b-9552-95545fe4a133\") " pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:47.870889 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.870854    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf6e7f19-79ab-456b-9552-95545fe4a133-plugins-config-volume\") pod \"payload-pre-processing-7964457696-g7kzk\" (UID: \"bf6e7f19-79ab-456b-9552-95545fe4a133\") " pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:47.871067 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.870894    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrqxq\" (UniqueName: \"kubernetes.io/projected/bf6e7f19-79ab-456b-9552-95545fe4a133-kube-api-access-vrqxq\") pod \"payload-pre-processing-7964457696-g7kzk\" (UID: \"bf6e7f19-79ab-456b-9552-95545fe4a133\") " pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:47.871528 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.871505    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf6e7f19-79ab-456b-9552-95545fe4a133-plugins-config-volume\") pod \"payload-pre-processing-7964457696-g7kzk\" (UID: \"bf6e7f19-79ab-456b-9552-95545fe4a133\") " pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:47.880785 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.880723    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrqxq\" (UniqueName: \"kubernetes.io/projected/bf6e7f19-79ab-456b-9552-95545fe4a133-kube-api-access-vrqxq\") pod \"payload-pre-processing-7964457696-g7kzk\" (UID: \"bf6e7f19-79ab-456b-9552-95545fe4a133\") " pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:47.943842 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:47.943792    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:48.078088 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:48.078056    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-g7kzk"]
Jul 10 13:36:48.080982 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:36:48.080953    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf6e7f19_79ab_456b_9552_95545fe4a133.slice/crio-b62b3ca3d76cade659dded82b4518ad7286d659293895428cc5ab8c1f2695056 WatchSource:0}: Error finding container b62b3ca3d76cade659dded82b4518ad7286d659293895428cc5ab8c1f2695056: Status 404 returned error can't find the container with id b62b3ca3d76cade659dded82b4518ad7286d659293895428cc5ab8c1f2695056
Jul 10 13:36:48.199106 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:48.199069    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g" event={"ID":"25af9b04-6c44-4828-a544-a4a06e5340c3","Type":"ContainerStarted","Data":"a3ca07eb8ef5eb6b56168b42a78a7f2e75e647f54e2b86c1d4f94e8c87c35c50"}
Jul 10 13:36:48.200473 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:48.200439    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk" event={"ID":"bf6e7f19-79ab-456b-9552-95545fe4a133","Type":"ContainerStarted","Data":"b62b3ca3d76cade659dded82b4518ad7286d659293895428cc5ab8c1f2695056"}
Jul 10 13:36:50.213448 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:50.213412    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g" event={"ID":"25af9b04-6c44-4828-a544-a4a06e5340c3","Type":"ContainerStarted","Data":"4109e10cc851d702838040ec86f18f04d160bc2382a0a6151e4d046c482081cf"}
Jul 10 13:36:50.213912 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:50.213521    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:50.214955 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:50.214928    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk" event={"ID":"bf6e7f19-79ab-456b-9552-95545fe4a133","Type":"ContainerStarted","Data":"7449cc0f4e186268149d36c367376da97941970c979f01eaaaf1f118c52b5716"}
Jul 10 13:36:50.215040 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:50.214982    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:36:50.232259 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:50.232219    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g" podStartSLOduration=1.9598379540000002 podStartE2EDuration="4.232205958s" totalImagesPullingTime="2.272368004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:36:50.231208121 +0000 UTC m=+634.097334775" watchObservedRunningTime="2026-07-10 13:36:50.232205958 +0000 UTC m=+634.098332611"
Jul 10 13:36:56.224870 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:56.224836    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84445c5967-8d52g"
Jul 10 13:36:56.244837 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:36:56.244786    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk" podStartSLOduration=7.652261215 podStartE2EDuration="9.24477117s" totalImagesPullingTime="1.592509955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:36:50.257343793 +0000 UTC m=+634.123470449" watchObservedRunningTime="2026-07-10 13:36:56.24477117 +0000 UTC m=+640.110897824"
Jul 10 13:37:01.221442 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:01.221406    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-g7kzk"
Jul 10 13:37:15.007330 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.007294    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"]
Jul 10 13:37:15.010544 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.010525    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.013333 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.013312    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 13:37:15.014412 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.014393    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 13:37:15.014544 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.014524    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 10 13:37:15.014608 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.014545    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ctsr9\""
Jul 10 13:37:15.023006 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.022977    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"]
Jul 10 13:37:15.086504 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.086473    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.086504 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.086507    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.086711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.086529    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.086711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.086545    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq76k\" (UniqueName: \"kubernetes.io/projected/6935803a-6131-4b13-aab7-8f2dd592ac55-kube-api-access-cq76k\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.086711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.086630    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.086711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.086658    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6935803a-6131-4b13-aab7-8f2dd592ac55-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187187 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187155    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187187 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187190    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6935803a-6131-4b13-aab7-8f2dd592ac55-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187236    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187253    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187277    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187301    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq76k\" (UniqueName: \"kubernetes.io/projected/6935803a-6131-4b13-aab7-8f2dd592ac55-kube-api-access-cq76k\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187634 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187596    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187711 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187670    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.187764 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.187712    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.189683 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.189658    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6935803a-6131-4b13-aab7-8f2dd592ac55-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.189903 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.189885    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6935803a-6131-4b13-aab7-8f2dd592ac55-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.197584 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.197560    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq76k\" (UniqueName: \"kubernetes.io/projected/6935803a-6131-4b13-aab7-8f2dd592ac55-kube-api-access-cq76k\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q\" (UID: \"6935803a-6131-4b13-aab7-8f2dd592ac55\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.321950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.321852    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:15.458788 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:15.458761    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"]
Jul 10 13:37:15.461466 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:37:15.461429    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6935803a_6131_4b13_aab7_8f2dd592ac55.slice/crio-2b6d405cb73dbd8d29baef5a9583323548cbbca779a2d971b000b699715d1c8a WatchSource:0}: Error finding container 2b6d405cb73dbd8d29baef5a9583323548cbbca779a2d971b000b699715d1c8a: Status 404 returned error can't find the container with id 2b6d405cb73dbd8d29baef5a9583323548cbbca779a2d971b000b699715d1c8a
Jul 10 13:37:16.331583 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:16.331547    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q" event={"ID":"6935803a-6131-4b13-aab7-8f2dd592ac55","Type":"ContainerStarted","Data":"2b6d405cb73dbd8d29baef5a9583323548cbbca779a2d971b000b699715d1c8a"}
Jul 10 13:37:17.337514 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:17.337478    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q" event={"ID":"6935803a-6131-4b13-aab7-8f2dd592ac55","Type":"ContainerStarted","Data":"be27147d6b7cfa70360ed3a19357bc98195a6864e2446d41530ac783e728b910"}
Jul 10 13:37:17.359656 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:17.359601    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q" podStartSLOduration=2.094273386 podStartE2EDuration="3.35958004s" totalImagesPullingTime="1.265306654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:37:17.357464939 +0000 UTC m=+661.223591593" watchObservedRunningTime="2026-07-10 13:37:17.35958004 +0000 UTC m=+661.225706695"
Jul 10 13:37:25.322922 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:25.322880    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:25.323345 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:25.322973    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:25.335633 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:25.335598    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:37:25.388083 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:37:25.388053    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q"
Jul 10 13:38:54.343294 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.343256    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f4d6fb599-c827b"]
Jul 10 13:38:54.346640 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.346623    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.354138 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.353800    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4d6fb599-c827b"]
Jul 10 13:38:54.489019 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.488991    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fd9d2fa-8ffe-41b5-83c2-f17d22df7008-tls-cert\") pod \"authorino-5f4d6fb599-c827b\" (UID: \"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008\") " pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.489019 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.489027    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4qws\" (UniqueName: \"kubernetes.io/projected/1fd9d2fa-8ffe-41b5-83c2-f17d22df7008-kube-api-access-c4qws\") pod \"authorino-5f4d6fb599-c827b\" (UID: \"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008\") " pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.590292 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.590265    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fd9d2fa-8ffe-41b5-83c2-f17d22df7008-tls-cert\") pod \"authorino-5f4d6fb599-c827b\" (UID: \"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008\") " pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.590422 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.590296    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4qws\" (UniqueName: \"kubernetes.io/projected/1fd9d2fa-8ffe-41b5-83c2-f17d22df7008-kube-api-access-c4qws\") pod \"authorino-5f4d6fb599-c827b\" (UID: \"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008\") " pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.592699 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.592674    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fd9d2fa-8ffe-41b5-83c2-f17d22df7008-tls-cert\") pod \"authorino-5f4d6fb599-c827b\" (UID: \"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008\") " pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.599721 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.599663    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4qws\" (UniqueName: \"kubernetes.io/projected/1fd9d2fa-8ffe-41b5-83c2-f17d22df7008-kube-api-access-c4qws\") pod \"authorino-5f4d6fb599-c827b\" (UID: \"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008\") " pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.656910 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.656885    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4d6fb599-c827b"
Jul 10 13:38:54.787489 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.787466    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4d6fb599-c827b"]
Jul 10 13:38:54.789963 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:38:54.789939    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fd9d2fa_8ffe_41b5_83c2_f17d22df7008.slice/crio-f8e37899bddf5b11ed65066ab5f3e0c9720622a5f4defff2326ba363ac281ba8 WatchSource:0}: Error finding container f8e37899bddf5b11ed65066ab5f3e0c9720622a5f4defff2326ba363ac281ba8: Status 404 returned error can't find the container with id f8e37899bddf5b11ed65066ab5f3e0c9720622a5f4defff2326ba363ac281ba8
Jul 10 13:38:54.791623 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:54.791606    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 13:38:55.783093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:55.783058    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4d6fb599-c827b" event={"ID":"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008","Type":"ContainerStarted","Data":"c2b49072586d29d7855b58e97f991a3335dbe70d0c5aa82ec51b6f3436f645c0"}
Jul 10 13:38:55.783474 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:55.783104    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4d6fb599-c827b" event={"ID":"1fd9d2fa-8ffe-41b5-83c2-f17d22df7008","Type":"ContainerStarted","Data":"f8e37899bddf5b11ed65066ab5f3e0c9720622a5f4defff2326ba363ac281ba8"}
Jul 10 13:38:55.809564 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:55.808032    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5f4d6fb599-c827b" podStartSLOduration=1.379276162 podStartE2EDuration="1.808009892s" totalImagesPullingTime="428.73373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:38:55.807572289 +0000 UTC m=+759.673698944" watchObservedRunningTime="2026-07-10 13:38:55.808009892 +0000 UTC m=+759.674136546"
Jul 10 13:38:55.852806 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:55.851275    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fdcb44678-mn6qx"]
Jul 10 13:38:55.852806 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:55.851500    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7fdcb44678-mn6qx" podUID="426d50a5-4e40-4e1a-b829-8df81808f086" containerName="authorino" containerID="cri-o://35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa" gracePeriod=30
Jul 10 13:38:56.118674 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.118646    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:38:56.305097 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.305057    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk99v\" (UniqueName: \"kubernetes.io/projected/426d50a5-4e40-4e1a-b829-8df81808f086-kube-api-access-zk99v\") pod \"426d50a5-4e40-4e1a-b829-8df81808f086\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") "
Jul 10 13:38:56.305275 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.305172    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/426d50a5-4e40-4e1a-b829-8df81808f086-tls-cert\") pod \"426d50a5-4e40-4e1a-b829-8df81808f086\" (UID: \"426d50a5-4e40-4e1a-b829-8df81808f086\") "
Jul 10 13:38:56.307187 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.307154    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426d50a5-4e40-4e1a-b829-8df81808f086-kube-api-access-zk99v" (OuterVolumeSpecName: "kube-api-access-zk99v") pod "426d50a5-4e40-4e1a-b829-8df81808f086" (UID: "426d50a5-4e40-4e1a-b829-8df81808f086"). InnerVolumeSpecName "kube-api-access-zk99v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:38:56.315254 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.315230    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/426d50a5-4e40-4e1a-b829-8df81808f086-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "426d50a5-4e40-4e1a-b829-8df81808f086" (UID: "426d50a5-4e40-4e1a-b829-8df81808f086"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:38:56.406011 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.405934    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk99v\" (UniqueName: \"kubernetes.io/projected/426d50a5-4e40-4e1a-b829-8df81808f086-kube-api-access-zk99v\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:38:56.406011 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.405963    2554 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/426d50a5-4e40-4e1a-b829-8df81808f086-tls-cert\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:38:56.789679 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.789642    2554 generic.go:356] "Generic (PLEG): container finished" podID="426d50a5-4e40-4e1a-b829-8df81808f086" containerID="35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa" exitCode=0
Jul 10 13:38:56.790094 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.789688    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdcb44678-mn6qx"
Jul 10 13:38:56.790094 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.789730    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdcb44678-mn6qx" event={"ID":"426d50a5-4e40-4e1a-b829-8df81808f086","Type":"ContainerDied","Data":"35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa"}
Jul 10 13:38:56.790094 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.789789    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdcb44678-mn6qx" event={"ID":"426d50a5-4e40-4e1a-b829-8df81808f086","Type":"ContainerDied","Data":"64a5b73fc5a83059cf4cf84555aac41446655a385ab0c4b0ef115580ae7b633f"}
Jul 10 13:38:56.790094 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.789841    2554 scope.go:117] "RemoveContainer" containerID="35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa"
Jul 10 13:38:56.798741 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.798711    2554 scope.go:117] "RemoveContainer" containerID="35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa"
Jul 10 13:38:56.799006 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:38:56.798987    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa\": container with ID starting with 35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa not found: ID does not exist" containerID="35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa"
Jul 10 13:38:56.799101 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.799013    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa"} err="failed to get container status \"35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa\": rpc error: code = NotFound desc = could not find container \"35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa\": container with ID starting with 35d5710b02d95f2d3b189cb9c551e0b5498afd365aa1f05b59def5b4abb107aa not found: ID does not exist"
Jul 10 13:38:56.816347 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.816318    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fdcb44678-mn6qx"]
Jul 10 13:38:56.824809 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:56.824783    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7fdcb44678-mn6qx"]
Jul 10 13:38:58.665209 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:38:58.665179    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426d50a5-4e40-4e1a-b829-8df81808f086" path="/var/lib/kubelet/pods/426d50a5-4e40-4e1a-b829-8df81808f086/volumes"
Jul 10 13:40:09.775384 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:09.775338    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-v5h7v"]
Jul 10 13:40:09.775862 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:09.775613    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v" podUID="996a92fc-c1a1-4b40-a747-903c753040c5" containerName="manager" containerID="cri-o://c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604" gracePeriod=30
Jul 10 13:40:10.016713 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.016690    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:40:10.086501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.086434    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/996a92fc-c1a1-4b40-a747-903c753040c5-webhook-cert\") pod \"996a92fc-c1a1-4b40-a747-903c753040c5\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") "
Jul 10 13:40:10.086501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.086495    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndfbd\" (UniqueName: \"kubernetes.io/projected/996a92fc-c1a1-4b40-a747-903c753040c5-kube-api-access-ndfbd\") pod \"996a92fc-c1a1-4b40-a747-903c753040c5\" (UID: \"996a92fc-c1a1-4b40-a747-903c753040c5\") "
Jul 10 13:40:10.088451 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.088425    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996a92fc-c1a1-4b40-a747-903c753040c5-kube-api-access-ndfbd" (OuterVolumeSpecName: "kube-api-access-ndfbd") pod "996a92fc-c1a1-4b40-a747-903c753040c5" (UID: "996a92fc-c1a1-4b40-a747-903c753040c5"). InnerVolumeSpecName "kube-api-access-ndfbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:40:10.088547 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.088476    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/996a92fc-c1a1-4b40-a747-903c753040c5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "996a92fc-c1a1-4b40-a747-903c753040c5" (UID: "996a92fc-c1a1-4b40-a747-903c753040c5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:40:10.129932 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.129901    2554 generic.go:356] "Generic (PLEG): container finished" podID="996a92fc-c1a1-4b40-a747-903c753040c5" containerID="c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604" exitCode=0
Jul 10 13:40:10.130044 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.129958    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v"
Jul 10 13:40:10.130044 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.129984    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v" event={"ID":"996a92fc-c1a1-4b40-a747-903c753040c5","Type":"ContainerDied","Data":"c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604"}
Jul 10 13:40:10.130130 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.130053    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5999cf9bb8-v5h7v" event={"ID":"996a92fc-c1a1-4b40-a747-903c753040c5","Type":"ContainerDied","Data":"de7db6f8ffe80db3ff764fc5d1669bdfb480ee734da7f0b89b616f50d5a429be"}
Jul 10 13:40:10.130130 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.130082    2554 scope.go:117] "RemoveContainer" containerID="c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604"
Jul 10 13:40:10.139743 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.139718    2554 scope.go:117] "RemoveContainer" containerID="c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604"
Jul 10 13:40:10.140021 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:40:10.139999    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604\": container with ID starting with c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604 not found: ID does not exist" containerID="c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604"
Jul 10 13:40:10.140092 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.140028    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604"} err="failed to get container status \"c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604\": rpc error: code = NotFound desc = could not find container \"c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604\": container with ID starting with c8a46e8f77a12de23547144f70b0626f6452d2b5884da61a6670233411ceb604 not found: ID does not exist"
Jul 10 13:40:10.153710 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.153689    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-v5h7v"]
Jul 10 13:40:10.158647 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.158626    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-v5h7v"]
Jul 10 13:40:10.187379 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.187354    2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/996a92fc-c1a1-4b40-a747-903c753040c5-webhook-cert\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:40:10.187459 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.187383    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndfbd\" (UniqueName: \"kubernetes.io/projected/996a92fc-c1a1-4b40-a747-903c753040c5-kube-api-access-ndfbd\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:40:10.663476 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:10.663444    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="996a92fc-c1a1-4b40-a747-903c753040c5" path="/var/lib/kubelet/pods/996a92fc-c1a1-4b40-a747-903c753040c5/volumes"
Jul 10 13:40:11.014693 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.014665    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-t6jnd"]
Jul 10 13:40:11.015081 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.015006    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426d50a5-4e40-4e1a-b829-8df81808f086" containerName="authorino"
Jul 10 13:40:11.015081 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.015018    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="426d50a5-4e40-4e1a-b829-8df81808f086" containerName="authorino"
Jul 10 13:40:11.015081 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.015032    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="996a92fc-c1a1-4b40-a747-903c753040c5" containerName="manager"
Jul 10 13:40:11.015081 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.015038    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="996a92fc-c1a1-4b40-a747-903c753040c5" containerName="manager"
Jul 10 13:40:11.015235 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.015093    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="426d50a5-4e40-4e1a-b829-8df81808f086" containerName="authorino"
Jul 10 13:40:11.015235 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.015103    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="996a92fc-c1a1-4b40-a747-903c753040c5" containerName="manager"
Jul 10 13:40:11.019605 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.019585    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.023350 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.023331    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 13:40:11.023463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.023375    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 13:40:11.023463 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.023330    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7pq9t\""
Jul 10 13:40:11.029088 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.029067    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-t6jnd"]
Jul 10 13:40:11.092603 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.092579    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2kpm\" (UniqueName: \"kubernetes.io/projected/150a898a-affd-4421-aab9-76ad94e1efd1-kube-api-access-v2kpm\") pod \"maas-controller-5999cf9bb8-t6jnd\" (UID: \"150a898a-affd-4421-aab9-76ad94e1efd1\") " pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.092725 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.092620    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150a898a-affd-4421-aab9-76ad94e1efd1-webhook-cert\") pod \"maas-controller-5999cf9bb8-t6jnd\" (UID: \"150a898a-affd-4421-aab9-76ad94e1efd1\") " pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.193481 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.193454    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2kpm\" (UniqueName: \"kubernetes.io/projected/150a898a-affd-4421-aab9-76ad94e1efd1-kube-api-access-v2kpm\") pod \"maas-controller-5999cf9bb8-t6jnd\" (UID: \"150a898a-affd-4421-aab9-76ad94e1efd1\") " pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.193586 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.193497    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150a898a-affd-4421-aab9-76ad94e1efd1-webhook-cert\") pod \"maas-controller-5999cf9bb8-t6jnd\" (UID: \"150a898a-affd-4421-aab9-76ad94e1efd1\") " pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.195866 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.195840    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150a898a-affd-4421-aab9-76ad94e1efd1-webhook-cert\") pod \"maas-controller-5999cf9bb8-t6jnd\" (UID: \"150a898a-affd-4421-aab9-76ad94e1efd1\") " pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.202989 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.202965    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2kpm\" (UniqueName: \"kubernetes.io/projected/150a898a-affd-4421-aab9-76ad94e1efd1-kube-api-access-v2kpm\") pod \"maas-controller-5999cf9bb8-t6jnd\" (UID: \"150a898a-affd-4421-aab9-76ad94e1efd1\") " pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.329667 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.329596    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:11.461556 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:11.461527    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5999cf9bb8-t6jnd"]
Jul 10 13:40:11.463496 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:40:11.463465    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150a898a_affd_4421_aab9_76ad94e1efd1.slice/crio-cf600a30cb07be1dd24de92b28bd70de3c720f6e86b3a39ed28e1b43b2df282e WatchSource:0}: Error finding container cf600a30cb07be1dd24de92b28bd70de3c720f6e86b3a39ed28e1b43b2df282e: Status 404 returned error can't find the container with id cf600a30cb07be1dd24de92b28bd70de3c720f6e86b3a39ed28e1b43b2df282e
Jul 10 13:40:12.141836 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:12.141791    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5999cf9bb8-t6jnd" event={"ID":"150a898a-affd-4421-aab9-76ad94e1efd1","Type":"ContainerStarted","Data":"d7a6c4de3cdcd418e64d1d27deb5fd96863cf1eb11317790d993f6cbed9cadf7"}
Jul 10 13:40:12.142199 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:12.141861    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5999cf9bb8-t6jnd" event={"ID":"150a898a-affd-4421-aab9-76ad94e1efd1","Type":"ContainerStarted","Data":"cf600a30cb07be1dd24de92b28bd70de3c720f6e86b3a39ed28e1b43b2df282e"}
Jul 10 13:40:12.142199 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:12.141887    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:40:12.162715 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:12.162664    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5999cf9bb8-t6jnd" podStartSLOduration=1.679956891 podStartE2EDuration="2.162648933s" totalImagesPullingTime="482.692042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:40:12.160608121 +0000 UTC m=+836.026734774" watchObservedRunningTime="2026-07-10 13:40:12.162648933 +0000 UTC m=+836.028775666"
Jul 10 13:40:23.152530 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:40:23.152501    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5999cf9bb8-t6jnd"
Jul 10 13:45:00.157696 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:00.157658    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"]
Jul 10 13:45:00.161360 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:00.161343    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"
Jul 10 13:45:00.165313 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:00.165284    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69k2\""
Jul 10 13:45:00.171030 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:00.171013    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"
Jul 10 13:45:00.228273 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:00.228242    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"]
Jul 10 13:45:00.712403 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:00.712381    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"]
Jul 10 13:45:00.714738 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:45:00.714712    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19da1986_4b52_42e2_9de4_cca96c610426.slice/crio-753e929ff21c30775988b0e4762d608d0c88990deaa1ff167c8d5a6ed1969f53 WatchSource:0}: Error finding container 753e929ff21c30775988b0e4762d608d0c88990deaa1ff167c8d5a6ed1969f53: Status 404 returned error can't find the container with id 753e929ff21c30775988b0e4762d608d0c88990deaa1ff167c8d5a6ed1969f53
Jul 10 13:45:00.717115 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:00.717099    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 13:45:01.469442 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:01.469408    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw" event={"ID":"19da1986-4b52-42e2-9de4-cca96c610426","Type":"ContainerStarted","Data":"753e929ff21c30775988b0e4762d608d0c88990deaa1ff167c8d5a6ed1969f53"}
Jul 10 13:45:03.481738 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:03.481703    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw" event={"ID":"19da1986-4b52-42e2-9de4-cca96c610426","Type":"ContainerStarted","Data":"044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b"}
Jul 10 13:45:03.501442 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:03.501393    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw" podStartSLOduration=1.156387373 podStartE2EDuration="3.501377647s" totalImagesPullingTime="2.344990274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:45:03.500322377 +0000 UTC m=+1127.366449032" watchObservedRunningTime="2026-07-10 13:45:03.501377647 +0000 UTC m=+1127.367504301"
Jul 10 13:45:04.157858 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:04.157799    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"]
Jul 10 13:45:04.158093 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:04.158063    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk" podUID="79991ff7-31df-4e6c-bbd7-f0a834044ce7" containerName="manager" containerID="cri-o://0637f55a86ca5ed80bac550fb1ff2b4cab1fd6efdda6fc8fe4c787c7f2d8a55e" gracePeriod=10
Jul 10 13:45:07.503457 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.503427    2554 generic.go:356] "Generic (PLEG): container finished" podID="79991ff7-31df-4e6c-bbd7-f0a834044ce7" containerID="0637f55a86ca5ed80bac550fb1ff2b4cab1fd6efdda6fc8fe4c787c7f2d8a55e" exitCode=0
Jul 10 13:45:07.503746 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.503487    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk" event={"ID":"79991ff7-31df-4e6c-bbd7-f0a834044ce7","Type":"ContainerDied","Data":"0637f55a86ca5ed80bac550fb1ff2b4cab1fd6efdda6fc8fe4c787c7f2d8a55e"}
Jul 10 13:45:07.503746 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.503530    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk" event={"ID":"79991ff7-31df-4e6c-bbd7-f0a834044ce7","Type":"ContainerDied","Data":"daec6ee277e4fa794558c045044da50d551e7045ab053632c0c70d860e8cb90d"}
Jul 10 13:45:07.503746 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.503547    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="daec6ee277e4fa794558c045044da50d551e7045ab053632c0c70d860e8cb90d"
Jul 10 13:45:07.511928 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.511906    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:45:07.618139 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.618073    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpq69\" (UniqueName: \"kubernetes.io/projected/79991ff7-31df-4e6c-bbd7-f0a834044ce7-kube-api-access-fpq69\") pod \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") "
Jul 10 13:45:07.618265 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.618221    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79991ff7-31df-4e6c-bbd7-f0a834044ce7-extensions-socket-volume\") pod \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\" (UID: \"79991ff7-31df-4e6c-bbd7-f0a834044ce7\") "
Jul 10 13:45:07.618583 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.618559    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79991ff7-31df-4e6c-bbd7-f0a834044ce7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "79991ff7-31df-4e6c-bbd7-f0a834044ce7" (UID: "79991ff7-31df-4e6c-bbd7-f0a834044ce7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:45:07.620101 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.620074    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79991ff7-31df-4e6c-bbd7-f0a834044ce7-kube-api-access-fpq69" (OuterVolumeSpecName: "kube-api-access-fpq69") pod "79991ff7-31df-4e6c-bbd7-f0a834044ce7" (UID: "79991ff7-31df-4e6c-bbd7-f0a834044ce7"). InnerVolumeSpecName "kube-api-access-fpq69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:45:07.718648 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.718626    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79991ff7-31df-4e6c-bbd7-f0a834044ce7-extensions-socket-volume\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:45:07.718648 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:07.718645    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpq69\" (UniqueName: \"kubernetes.io/projected/79991ff7-31df-4e6c-bbd7-f0a834044ce7-kube-api-access-fpq69\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:45:08.507897 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:08.507870    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"
Jul 10 13:45:08.532999 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:08.532975    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"]
Jul 10 13:45:08.538343 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:08.538323    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gqswk"]
Jul 10 13:45:08.663658 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:08.663633    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79991ff7-31df-4e6c-bbd7-f0a834044ce7" path="/var/lib/kubelet/pods/79991ff7-31df-4e6c-bbd7-f0a834044ce7/volumes"
Jul 10 13:45:16.734487 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:45:16.734455    2554 scope.go:117] "RemoveContainer" containerID="0637f55a86ca5ed80bac550fb1ff2b4cab1fd6efdda6fc8fe4c787c7f2d8a55e"
Jul 10 13:46:00.015477 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:00.015267    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"]
Jul 10 13:46:00.016020 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:00.015613    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw" podUID="19da1986-4b52-42e2-9de4-cca96c610426" containerName="cleanup" containerID="cri-o://044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b" gracePeriod=30
Jul 10 13:46:05.372163 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.372129    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"]
Jul 10 13:46:05.372535 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.372469    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79991ff7-31df-4e6c-bbd7-f0a834044ce7" containerName="manager"
Jul 10 13:46:05.372535 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.372480    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="79991ff7-31df-4e6c-bbd7-f0a834044ce7" containerName="manager"
Jul 10 13:46:05.372606 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.372549    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="79991ff7-31df-4e6c-bbd7-f0a834044ce7" containerName="manager"
Jul 10 13:46:05.375694 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.375673    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.379169 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.379142    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cgx88\""
Jul 10 13:46:05.389282 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.389255    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"]
Jul 10 13:46:05.541728 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.541695    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9wfb\" (UniqueName: \"kubernetes.io/projected/f07c6756-fbde-41cd-836d-7d54d13f62a9-kube-api-access-g9wfb\") pod \"kuadrant-operator-controller-manager-86878c769f-fjvw9\" (UID: \"f07c6756-fbde-41cd-836d-7d54d13f62a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.541971 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.541750    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f07c6756-fbde-41cd-836d-7d54d13f62a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjvw9\" (UID: \"f07c6756-fbde-41cd-836d-7d54d13f62a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.642586 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.642504    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f07c6756-fbde-41cd-836d-7d54d13f62a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjvw9\" (UID: \"f07c6756-fbde-41cd-836d-7d54d13f62a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.642586 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.642571    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9wfb\" (UniqueName: \"kubernetes.io/projected/f07c6756-fbde-41cd-836d-7d54d13f62a9-kube-api-access-g9wfb\") pod \"kuadrant-operator-controller-manager-86878c769f-fjvw9\" (UID: \"f07c6756-fbde-41cd-836d-7d54d13f62a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.642889 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.642868    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f07c6756-fbde-41cd-836d-7d54d13f62a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjvw9\" (UID: \"f07c6756-fbde-41cd-836d-7d54d13f62a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.652852 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.652815    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9wfb\" (UniqueName: \"kubernetes.io/projected/f07c6756-fbde-41cd-836d-7d54d13f62a9-kube-api-access-g9wfb\") pod \"kuadrant-operator-controller-manager-86878c769f-fjvw9\" (UID: \"f07c6756-fbde-41cd-836d-7d54d13f62a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.687740 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.687721    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:05.830248 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:05.830226    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"]
Jul 10 13:46:05.833068 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:46:05.833043    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07c6756_fbde_41cd_836d_7d54d13f62a9.slice/crio-6778c1ee427047a1011ebf05e063c7f92088d389998121f7922f99984d7aba3d WatchSource:0}: Error finding container 6778c1ee427047a1011ebf05e063c7f92088d389998121f7922f99984d7aba3d: Status 404 returned error can't find the container with id 6778c1ee427047a1011ebf05e063c7f92088d389998121f7922f99984d7aba3d
Jul 10 13:46:06.769937 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:06.769901    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9" event={"ID":"f07c6756-fbde-41cd-836d-7d54d13f62a9","Type":"ContainerStarted","Data":"81d81cbd347ad82219d535c34b1004246d7dc3c18c38d9b81b619840094f587c"}
Jul 10 13:46:06.770302 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:06.769956    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9" event={"ID":"f07c6756-fbde-41cd-836d-7d54d13f62a9","Type":"ContainerStarted","Data":"6778c1ee427047a1011ebf05e063c7f92088d389998121f7922f99984d7aba3d"}
Jul 10 13:46:06.770302 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:06.770050    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:06.793014 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:06.792974    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9" podStartSLOduration=1.7929575359999999 podStartE2EDuration="1.792957536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:46:06.791140999 +0000 UTC m=+1190.657267652" watchObservedRunningTime="2026-07-10 13:46:06.792957536 +0000 UTC m=+1190.659084234"
Jul 10 13:46:17.776608 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:17.776574    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjvw9"
Jul 10 13:46:30.201498 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.201475    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"
Jul 10 13:46:30.889701 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.889664    2554 generic.go:356] "Generic (PLEG): container finished" podID="19da1986-4b52-42e2-9de4-cca96c610426" containerID="044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b" exitCode=137
Jul 10 13:46:30.889950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.889712    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"
Jul 10 13:46:30.889950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.889759    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw" event={"ID":"19da1986-4b52-42e2-9de4-cca96c610426","Type":"ContainerDied","Data":"044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b"}
Jul 10 13:46:30.889950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.889835    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw" event={"ID":"19da1986-4b52-42e2-9de4-cca96c610426","Type":"ContainerDied","Data":"753e929ff21c30775988b0e4762d608d0c88990deaa1ff167c8d5a6ed1969f53"}
Jul 10 13:46:30.889950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.889868    2554 scope.go:117] "RemoveContainer" containerID="044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b"
Jul 10 13:46:30.898305 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.898290    2554 scope.go:117] "RemoveContainer" containerID="044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b"
Jul 10 13:46:30.898583 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:46:30.898565    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b\": container with ID starting with 044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b not found: ID does not exist" containerID="044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b"
Jul 10 13:46:30.898645 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.898591    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b"} err="failed to get container status \"044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b\": rpc error: code = NotFound desc = could not find container \"044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b\": container with ID starting with 044f6ac543e53424e30aaf45d6f076824ec38a3542a0e846b0802bc774e72c3b not found: ID does not exist"
Jul 10 13:46:30.910772 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.910750    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"]
Jul 10 13:46:30.914518 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:30.914500    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728185-xzcpw"]
Jul 10 13:46:32.663595 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:46:32.663564    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19da1986-4b52-42e2-9de4-cca96c610426" path="/var/lib/kubelet/pods/19da1986-4b52-42e2-9de4-cca96c610426/volumes"
Jul 10 13:52:13.826301 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.826264    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"]
Jul 10 13:52:13.826715 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.826684    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19da1986-4b52-42e2-9de4-cca96c610426" containerName="cleanup"
Jul 10 13:52:13.826715 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.826701    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="19da1986-4b52-42e2-9de4-cca96c610426" containerName="cleanup"
Jul 10 13:52:13.826798 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.826779    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="19da1986-4b52-42e2-9de4-cca96c610426" containerName="cleanup"
Jul 10 13:52:13.830129 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.830110    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:13.832559 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.832542    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-283dfe49-serving-cert\""
Jul 10 13:52:13.839262 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.839243    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"]
Jul 10 13:52:13.939163 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.939133    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d1887d6-7800-424b-a21d-02caba185e2c-openshift-service-ca\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:13.939315 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.939177    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5v9v\" (UniqueName: \"kubernetes.io/projected/4d1887d6-7800-424b-a21d-02caba185e2c-kube-api-access-l5v9v\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:13.939315 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:13.939193    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d1887d6-7800-424b-a21d-02caba185e2c-maas-api-tls\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.039936 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.039903    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d1887d6-7800-424b-a21d-02caba185e2c-openshift-service-ca\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.040066 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.039948    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5v9v\" (UniqueName: \"kubernetes.io/projected/4d1887d6-7800-424b-a21d-02caba185e2c-kube-api-access-l5v9v\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.040066 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.039972    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d1887d6-7800-424b-a21d-02caba185e2c-maas-api-tls\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.040721 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.040695    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d1887d6-7800-424b-a21d-02caba185e2c-openshift-service-ca\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.042437 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.042416    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d1887d6-7800-424b-a21d-02caba185e2c-maas-api-tls\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.053628 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.053593    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5v9v\" (UniqueName: \"kubernetes.io/projected/4d1887d6-7800-424b-a21d-02caba185e2c-kube-api-access-l5v9v\") pod \"maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.142982 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.142917    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:14.294110 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.294067    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"]
Jul 10 13:52:14.296145 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:52:14.296120    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d1887d6_7800_424b_a21d_02caba185e2c.slice/crio-9228aecc87edc43025c889a3afdd9e253e22888ab97078b740f1507e30d0165a WatchSource:0}: Error finding container 9228aecc87edc43025c889a3afdd9e253e22888ab97078b740f1507e30d0165a: Status 404 returned error can't find the container with id 9228aecc87edc43025c889a3afdd9e253e22888ab97078b740f1507e30d0165a
Jul 10 13:52:14.297512 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.297492    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 13:52:14.445319 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:14.445288    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k" event={"ID":"4d1887d6-7800-424b-a21d-02caba185e2c","Type":"ContainerStarted","Data":"9228aecc87edc43025c889a3afdd9e253e22888ab97078b740f1507e30d0165a"}
Jul 10 13:52:15.451975 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:15.451940    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k" event={"ID":"4d1887d6-7800-424b-a21d-02caba185e2c","Type":"ContainerStarted","Data":"8003ba4b5bdd40b7a6b492d70fa6fde67fabb6cb84e874bf1ddf01447aa94f3a"}
Jul 10 13:52:15.452570 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:15.452541    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:15.470853 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:15.470791    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k" podStartSLOduration=1.910418918 podStartE2EDuration="2.470775689s" totalImagesPullingTime="560.356771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:52:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:52:15.469586497 +0000 UTC m=+1559.335713152" watchObservedRunningTime="2026-07-10 13:52:15.470775689 +0000 UTC m=+1559.336902343"
Jul 10 13:52:19.277678 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.277645    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"]
Jul 10 13:52:19.278402 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.277893    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k" podUID="4d1887d6-7800-424b-a21d-02caba185e2c" containerName="maas-api" containerID="cri-o://8003ba4b5bdd40b7a6b492d70fa6fde67fabb6cb84e874bf1ddf01447aa94f3a" gracePeriod=30
Jul 10 13:52:19.283838 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.283796    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:19.472666 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.472629    2554 generic.go:356] "Generic (PLEG): container finished" podID="4d1887d6-7800-424b-a21d-02caba185e2c" containerID="8003ba4b5bdd40b7a6b492d70fa6fde67fabb6cb84e874bf1ddf01447aa94f3a" exitCode=0
Jul 10 13:52:19.472807 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.472681    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k" event={"ID":"4d1887d6-7800-424b-a21d-02caba185e2c","Type":"ContainerDied","Data":"8003ba4b5bdd40b7a6b492d70fa6fde67fabb6cb84e874bf1ddf01447aa94f3a"}
Jul 10 13:52:19.525781 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.525760    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:19.588203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.588136    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5v9v\" (UniqueName: \"kubernetes.io/projected/4d1887d6-7800-424b-a21d-02caba185e2c-kube-api-access-l5v9v\") pod \"4d1887d6-7800-424b-a21d-02caba185e2c\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") "
Jul 10 13:52:19.588203 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.588201    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d1887d6-7800-424b-a21d-02caba185e2c-openshift-service-ca\") pod \"4d1887d6-7800-424b-a21d-02caba185e2c\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") "
Jul 10 13:52:19.588389 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.588245    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d1887d6-7800-424b-a21d-02caba185e2c-maas-api-tls\") pod \"4d1887d6-7800-424b-a21d-02caba185e2c\" (UID: \"4d1887d6-7800-424b-a21d-02caba185e2c\") "
Jul 10 13:52:19.588515 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.588493    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d1887d6-7800-424b-a21d-02caba185e2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d1887d6-7800-424b-a21d-02caba185e2c" (UID: "4d1887d6-7800-424b-a21d-02caba185e2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:52:19.590346 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.590324    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d1887d6-7800-424b-a21d-02caba185e2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d1887d6-7800-424b-a21d-02caba185e2c" (UID: "4d1887d6-7800-424b-a21d-02caba185e2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:52:19.590438 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.590320    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d1887d6-7800-424b-a21d-02caba185e2c-kube-api-access-l5v9v" (OuterVolumeSpecName: "kube-api-access-l5v9v") pod "4d1887d6-7800-424b-a21d-02caba185e2c" (UID: "4d1887d6-7800-424b-a21d-02caba185e2c"). InnerVolumeSpecName "kube-api-access-l5v9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:52:19.689576 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.689551    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d1887d6-7800-424b-a21d-02caba185e2c-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:19.689677 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.689580    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d1887d6-7800-424b-a21d-02caba185e2c-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:19.689677 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:19.689592    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5v9v\" (UniqueName: \"kubernetes.io/projected/4d1887d6-7800-424b-a21d-02caba185e2c-kube-api-access-l5v9v\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:20.479172 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:20.479146    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"
Jul 10 13:52:20.479610 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:20.479189    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k" event={"ID":"4d1887d6-7800-424b-a21d-02caba185e2c","Type":"ContainerDied","Data":"9228aecc87edc43025c889a3afdd9e253e22888ab97078b740f1507e30d0165a"}
Jul 10 13:52:20.479610 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:20.479254    2554 scope.go:117] "RemoveContainer" containerID="8003ba4b5bdd40b7a6b492d70fa6fde67fabb6cb84e874bf1ddf01447aa94f3a"
Jul 10 13:52:20.502858 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:20.502837    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"]
Jul 10 13:52:20.508349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:20.508324    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-283dfe49-5748d56bc-6kr9k"]
Jul 10 13:52:20.664160 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:20.664134    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d1887d6-7800-424b-a21d-02caba185e2c" path="/var/lib/kubelet/pods/4d1887d6-7800-424b-a21d-02caba185e2c/volumes"
Jul 10 13:52:27.269669 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.269638    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"]
Jul 10 13:52:27.270195 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.270068    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d1887d6-7800-424b-a21d-02caba185e2c" containerName="maas-api"
Jul 10 13:52:27.270195 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.270085    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d1887d6-7800-424b-a21d-02caba185e2c" containerName="maas-api"
Jul 10 13:52:27.270195 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.270167    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d1887d6-7800-424b-a21d-02caba185e2c" containerName="maas-api"
Jul 10 13:52:27.285842 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.285785    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"]
Jul 10 13:52:27.285984 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.285918    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.288445 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.288421    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f2237c2f-serving-cert\""
Jul 10 13:52:27.347359 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.347335    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.347501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.347373    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-openshift-service-ca\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.347501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.347442    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5p4c\" (UniqueName: \"kubernetes.io/projected/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-kube-api-access-g5p4c\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.448572 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.448546    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.448700 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.448583    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-openshift-service-ca\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.448700 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.448618    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5p4c\" (UniqueName: \"kubernetes.io/projected/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-kube-api-access-g5p4c\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.448837 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:52:27.448701    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-serving-cert: secret "maas-api-e2e-ait-f2237c2f-serving-cert" not found
Jul 10 13:52:27.448837 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:52:27.448785    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls podName:aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5 nodeName:}" failed. No retries permitted until 2026-07-10 13:52:27.948764046 +0000 UTC m=+1571.814890698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls") pod "maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd" (UID: "aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5") : secret "maas-api-e2e-ait-f2237c2f-serving-cert" not found
Jul 10 13:52:27.449294 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.449275    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-openshift-service-ca\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.461843 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.461804    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5p4c\" (UniqueName: \"kubernetes.io/projected/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-kube-api-access-g5p4c\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.952725 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.952680    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:27.955799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:27.955766    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls\") pod \"maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:28.197433 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:28.197398    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:28.336698 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:28.336654    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"]
Jul 10 13:52:28.339476 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:52:28.339434    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa64836a_b8d7_4ccf_b66e_8feb37e8b3b5.slice/crio-508ea4043f9210c9b95ab846fdd9db28f7c9b658dda77950ce74a6476684f9ab WatchSource:0}: Error finding container 508ea4043f9210c9b95ab846fdd9db28f7c9b658dda77950ce74a6476684f9ab: Status 404 returned error can't find the container with id 508ea4043f9210c9b95ab846fdd9db28f7c9b658dda77950ce74a6476684f9ab
Jul 10 13:52:28.518752 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:28.518659    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd" event={"ID":"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5","Type":"ContainerStarted","Data":"508ea4043f9210c9b95ab846fdd9db28f7c9b658dda77950ce74a6476684f9ab"}
Jul 10 13:52:29.524739 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:29.524711    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd" event={"ID":"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5","Type":"ContainerStarted","Data":"57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e"}
Jul 10 13:52:29.525133 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:29.524812    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:29.547300 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:29.547246    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd" podStartSLOduration=2.14143115 podStartE2EDuration="2.547232069s" totalImagesPullingTime="405.800919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:52:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:52:29.545500534 +0000 UTC m=+1573.411627189" watchObservedRunningTime="2026-07-10 13:52:29.547232069 +0000 UTC m=+1573.413358722"
Jul 10 13:52:32.714353 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:32.714316    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"]
Jul 10 13:52:32.714796 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:32.714509    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd" podUID="aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" containerName="maas-api" containerID="cri-o://57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e" gracePeriod=30
Jul 10 13:52:32.964853 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:32.964784    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:33.090031 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.090007    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls\") pod \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") "
Jul 10 13:52:33.090174 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.090053    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-openshift-service-ca\") pod \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") "
Jul 10 13:52:33.090174 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.090086    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5p4c\" (UniqueName: \"kubernetes.io/projected/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-kube-api-access-g5p4c\") pod \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\" (UID: \"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5\") "
Jul 10 13:52:33.090465 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.090435    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" (UID: "aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:52:33.092023 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.092000    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" (UID: "aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:52:33.092159 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.092138    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-kube-api-access-g5p4c" (OuterVolumeSpecName: "kube-api-access-g5p4c") pod "aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" (UID: "aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5"). InnerVolumeSpecName "kube-api-access-g5p4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:52:33.191688 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.191663    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:33.191807 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.191685    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:33.191807 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.191707    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5p4c\" (UniqueName: \"kubernetes.io/projected/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5-kube-api-access-g5p4c\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:33.546830 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.546794    2554 generic.go:356] "Generic (PLEG): container finished" podID="aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" containerID="57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e" exitCode=0
Jul 10 13:52:33.546995 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.546852    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"
Jul 10 13:52:33.546995 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.546889    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd" event={"ID":"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5","Type":"ContainerDied","Data":"57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e"}
Jul 10 13:52:33.546995 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.546944    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd" event={"ID":"aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5","Type":"ContainerDied","Data":"508ea4043f9210c9b95ab846fdd9db28f7c9b658dda77950ce74a6476684f9ab"}
Jul 10 13:52:33.546995 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.546969    2554 scope.go:117] "RemoveContainer" containerID="57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e"
Jul 10 13:52:33.555773 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.555757    2554 scope.go:117] "RemoveContainer" containerID="57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e"
Jul 10 13:52:33.556052 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:52:33.556033    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e\": container with ID starting with 57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e not found: ID does not exist" containerID="57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e"
Jul 10 13:52:33.556099 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.556062    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e"} err="failed to get container status \"57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e\": rpc error: code = NotFound desc = could not find container \"57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e\": container with ID starting with 57c8952e9ae03772a44e8e6e40a1152129ad27efdbc130a50c854b30028a248e not found: ID does not exist"
Jul 10 13:52:33.569175 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.569157    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"]
Jul 10 13:52:33.574430 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:33.574411    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f2237c2f-5554d57bc7-q28sd"]
Jul 10 13:52:34.665008 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:34.664974    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" path="/var/lib/kubelet/pods/aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5/volumes"
Jul 10 13:52:46.487962 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.487930    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"]
Jul 10 13:52:46.488399 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.488366    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" containerName="maas-api"
Jul 10 13:52:46.488399 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.488379    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" containerName="maas-api"
Jul 10 13:52:46.488519 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.488447    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa64836a-b8d7-4ccf-b66e-8feb37e8b3b5" containerName="maas-api"
Jul 10 13:52:46.491260 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.491240    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.493950 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.493924    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d5a86b34-serving-cert\""
Jul 10 13:52:46.501004 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.500981    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"]
Jul 10 13:52:46.591003 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.590974    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmvvk\" (UniqueName: \"kubernetes.io/projected/b43adde2-c127-4614-9bc6-446d195c8312-kube-api-access-wmvvk\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.591137 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.591010    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.591137 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.591039    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43adde2-c127-4614-9bc6-446d195c8312-openshift-service-ca\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.692138 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.692101    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmvvk\" (UniqueName: \"kubernetes.io/projected/b43adde2-c127-4614-9bc6-446d195c8312-kube-api-access-wmvvk\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.692267 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.692143    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.692267 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.692176    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43adde2-c127-4614-9bc6-446d195c8312-openshift-service-ca\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.692345 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:52:46.692312    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-serving-cert: secret "maas-api-e2e-derive-d5a86b34-serving-cert" not found
Jul 10 13:52:46.692384 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:52:46.692372    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls podName:b43adde2-c127-4614-9bc6-446d195c8312 nodeName:}" failed. No retries permitted until 2026-07-10 13:52:47.192354868 +0000 UTC m=+1591.058481500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls") pod "maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn" (UID: "b43adde2-c127-4614-9bc6-446d195c8312") : secret "maas-api-e2e-derive-d5a86b34-serving-cert" not found
Jul 10 13:52:46.692945 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.692926    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43adde2-c127-4614-9bc6-446d195c8312-openshift-service-ca\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:46.704927 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:46.704908    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmvvk\" (UniqueName: \"kubernetes.io/projected/b43adde2-c127-4614-9bc6-446d195c8312-kube-api-access-wmvvk\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:47.195367 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:47.195333    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:47.197639 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:47.197610    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls\") pod \"maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:47.402691 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:47.402656    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:47.552108 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:47.552026    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"]
Jul 10 13:52:47.555420 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:52:47.555391    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43adde2_c127_4614_9bc6_446d195c8312.slice/crio-e8b892bf0ef35b575f0f5b1fec91cb9a5fba352d4cec14e4d8e826e411048e80 WatchSource:0}: Error finding container e8b892bf0ef35b575f0f5b1fec91cb9a5fba352d4cec14e4d8e826e411048e80: Status 404 returned error can't find the container with id e8b892bf0ef35b575f0f5b1fec91cb9a5fba352d4cec14e4d8e826e411048e80
Jul 10 13:52:47.614606 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:47.614575    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn" event={"ID":"b43adde2-c127-4614-9bc6-446d195c8312","Type":"ContainerStarted","Data":"e8b892bf0ef35b575f0f5b1fec91cb9a5fba352d4cec14e4d8e826e411048e80"}
Jul 10 13:52:48.620692 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:48.620658    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn" event={"ID":"b43adde2-c127-4614-9bc6-446d195c8312","Type":"ContainerStarted","Data":"cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22"}
Jul 10 13:52:48.621068 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:48.620784    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:48.639389 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:48.639349    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn" podStartSLOduration=2.279079066 podStartE2EDuration="2.639335064s" totalImagesPullingTime="360.255998ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:52:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:52:48.638226738 +0000 UTC m=+1592.504353393" watchObservedRunningTime="2026-07-10 13:52:48.639335064 +0000 UTC m=+1592.505461717"
Jul 10 13:52:51.148233 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.148172    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"]
Jul 10 13:52:51.149426 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.148771    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn" podUID="b43adde2-c127-4614-9bc6-446d195c8312" containerName="maas-api" containerID="cri-o://cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22" gracePeriod=30
Jul 10 13:52:51.409752 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.409727    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:51.528982 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.528914    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls\") pod \"b43adde2-c127-4614-9bc6-446d195c8312\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") "
Jul 10 13:52:51.528982 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.528949    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmvvk\" (UniqueName: \"kubernetes.io/projected/b43adde2-c127-4614-9bc6-446d195c8312-kube-api-access-wmvvk\") pod \"b43adde2-c127-4614-9bc6-446d195c8312\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") "
Jul 10 13:52:51.529182 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.528986    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43adde2-c127-4614-9bc6-446d195c8312-openshift-service-ca\") pod \"b43adde2-c127-4614-9bc6-446d195c8312\" (UID: \"b43adde2-c127-4614-9bc6-446d195c8312\") "
Jul 10 13:52:51.529486 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.529456    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43adde2-c127-4614-9bc6-446d195c8312-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b43adde2-c127-4614-9bc6-446d195c8312" (UID: "b43adde2-c127-4614-9bc6-446d195c8312"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:52:51.530947 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.530924    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b43adde2-c127-4614-9bc6-446d195c8312-kube-api-access-wmvvk" (OuterVolumeSpecName: "kube-api-access-wmvvk") pod "b43adde2-c127-4614-9bc6-446d195c8312" (UID: "b43adde2-c127-4614-9bc6-446d195c8312"). InnerVolumeSpecName "kube-api-access-wmvvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:52:51.531029 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.530925    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b43adde2-c127-4614-9bc6-446d195c8312" (UID: "b43adde2-c127-4614-9bc6-446d195c8312"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:52:51.630054 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.630023    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b43adde2-c127-4614-9bc6-446d195c8312-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:51.630054 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.630050    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmvvk\" (UniqueName: \"kubernetes.io/projected/b43adde2-c127-4614-9bc6-446d195c8312-kube-api-access-wmvvk\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:51.630193 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.630064    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43adde2-c127-4614-9bc6-446d195c8312-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:52:51.635898 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.635858    2554 generic.go:356] "Generic (PLEG): container finished" podID="b43adde2-c127-4614-9bc6-446d195c8312" containerID="cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22" exitCode=0
Jul 10 13:52:51.635976 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.635901    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"
Jul 10 13:52:51.635976 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.635951    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn" event={"ID":"b43adde2-c127-4614-9bc6-446d195c8312","Type":"ContainerDied","Data":"cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22"}
Jul 10 13:52:51.636042 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.636005    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn" event={"ID":"b43adde2-c127-4614-9bc6-446d195c8312","Type":"ContainerDied","Data":"e8b892bf0ef35b575f0f5b1fec91cb9a5fba352d4cec14e4d8e826e411048e80"}
Jul 10 13:52:51.636042 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.636032    2554 scope.go:117] "RemoveContainer" containerID="cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22"
Jul 10 13:52:51.644652 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.644632    2554 scope.go:117] "RemoveContainer" containerID="cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22"
Jul 10 13:52:51.644926 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:52:51.644907    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22\": container with ID starting with cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22 not found: ID does not exist" containerID="cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22"
Jul 10 13:52:51.644978 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.644934    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22"} err="failed to get container status \"cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22\": rpc error: code = NotFound desc = could not find container \"cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22\": container with ID starting with cabf0b985ab7bc267ef0b5a4dca052bf22a173c48b62398db98f1827fa48fd22 not found: ID does not exist"
Jul 10 13:52:51.659173 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.659151    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"]
Jul 10 13:52:51.664079 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:51.664059    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d5a86b34-6c9c48bbc8-nd2dn"]
Jul 10 13:52:52.670377 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:52:52.670345    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b43adde2-c127-4614-9bc6-446d195c8312" path="/var/lib/kubelet/pods/b43adde2-c127-4614-9bc6-446d195c8312/volumes"
Jul 10 13:54:42.028600 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.028522    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"]
Jul 10 13:54:42.029018 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.028879    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43adde2-c127-4614-9bc6-446d195c8312" containerName="maas-api"
Jul 10 13:54:42.029018 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.028892    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43adde2-c127-4614-9bc6-446d195c8312" containerName="maas-api"
Jul 10 13:54:42.029018 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.028950    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b43adde2-c127-4614-9bc6-446d195c8312" containerName="maas-api"
Jul 10 13:54:42.031958 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.031932    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.034569 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.034550    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d792a168-serving-cert\""
Jul 10 13:54:42.048285 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.048257    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"]
Jul 10 13:54:42.069713 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.069687    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.069843 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.069770    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28699a1d-d60d-4b66-9fe6-1327f52f585e-openshift-service-ca\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.069843 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.069798    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klw5t\" (UniqueName: \"kubernetes.io/projected/28699a1d-d60d-4b66-9fe6-1327f52f585e-kube-api-access-klw5t\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.171192 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.171157    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28699a1d-d60d-4b66-9fe6-1327f52f585e-openshift-service-ca\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.171192 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.171192    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klw5t\" (UniqueName: \"kubernetes.io/projected/28699a1d-d60d-4b66-9fe6-1327f52f585e-kube-api-access-klw5t\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.171443 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.171229    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.171443 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:54:42.171343    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-serving-cert: secret "maas-api-e2e-mt-d792a168-serving-cert" not found
Jul 10 13:54:42.171443 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:54:42.171410    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls podName:28699a1d-d60d-4b66-9fe6-1327f52f585e nodeName:}" failed. No retries permitted until 2026-07-10 13:54:42.671392103 +0000 UTC m=+1706.537518737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls") pod "maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w" (UID: "28699a1d-d60d-4b66-9fe6-1327f52f585e") : secret "maas-api-e2e-mt-d792a168-serving-cert" not found
Jul 10 13:54:42.171789 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.171770    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28699a1d-d60d-4b66-9fe6-1327f52f585e-openshift-service-ca\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.180470 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.180437    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klw5t\" (UniqueName: \"kubernetes.io/projected/28699a1d-d60d-4b66-9fe6-1327f52f585e-kube-api-access-klw5t\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.676151 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.676121    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.678537 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.678518    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls\") pod \"maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:42.950002 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:42.949909    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:43.090354 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:43.090323    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"]
Jul 10 13:54:43.092284 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:54:43.092260    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28699a1d_d60d_4b66_9fe6_1327f52f585e.slice/crio-995cc428dd9c25394b45274b836ed6168c6597facba424917f9a450a53d26c09 WatchSource:0}: Error finding container 995cc428dd9c25394b45274b836ed6168c6597facba424917f9a450a53d26c09: Status 404 returned error can't find the container with id 995cc428dd9c25394b45274b836ed6168c6597facba424917f9a450a53d26c09
Jul 10 13:54:43.150090 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:43.150056    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w" event={"ID":"28699a1d-d60d-4b66-9fe6-1327f52f585e","Type":"ContainerStarted","Data":"995cc428dd9c25394b45274b836ed6168c6597facba424917f9a450a53d26c09"}
Jul 10 13:54:44.156575 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:44.156538    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w" event={"ID":"28699a1d-d60d-4b66-9fe6-1327f52f585e","Type":"ContainerStarted","Data":"1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1"}
Jul 10 13:54:44.157005 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:44.156681    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:44.187808 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:44.187762    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w" podStartSLOduration=1.73666317 podStartE2EDuration="2.187748846s" totalImagesPullingTime="451.085676ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:54:44.185716431 +0000 UTC m=+1708.051843086" watchObservedRunningTime="2026-07-10 13:54:44.187748846 +0000 UTC m=+1708.053875500"
Jul 10 13:54:50.167341 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:50.167312    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:54.725403 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:54.725372    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"]
Jul 10 13:54:54.727704 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:54.725570    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w" podUID="28699a1d-d60d-4b66-9fe6-1327f52f585e" containerName="maas-api" containerID="cri-o://1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1" gracePeriod=30
Jul 10 13:54:54.978792 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:54.978737    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:55.072800 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.072773    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls\") pod \"28699a1d-d60d-4b66-9fe6-1327f52f585e\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") "
Jul 10 13:54:55.072984 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.072868    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28699a1d-d60d-4b66-9fe6-1327f52f585e-openshift-service-ca\") pod \"28699a1d-d60d-4b66-9fe6-1327f52f585e\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") "
Jul 10 13:54:55.072984 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.072900    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klw5t\" (UniqueName: \"kubernetes.io/projected/28699a1d-d60d-4b66-9fe6-1327f52f585e-kube-api-access-klw5t\") pod \"28699a1d-d60d-4b66-9fe6-1327f52f585e\" (UID: \"28699a1d-d60d-4b66-9fe6-1327f52f585e\") "
Jul 10 13:54:55.073186 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.073164    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28699a1d-d60d-4b66-9fe6-1327f52f585e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28699a1d-d60d-4b66-9fe6-1327f52f585e" (UID: "28699a1d-d60d-4b66-9fe6-1327f52f585e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:54:55.074812 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.074779    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28699a1d-d60d-4b66-9fe6-1327f52f585e" (UID: "28699a1d-d60d-4b66-9fe6-1327f52f585e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:54:55.074937 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.074888    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28699a1d-d60d-4b66-9fe6-1327f52f585e-kube-api-access-klw5t" (OuterVolumeSpecName: "kube-api-access-klw5t") pod "28699a1d-d60d-4b66-9fe6-1327f52f585e" (UID: "28699a1d-d60d-4b66-9fe6-1327f52f585e"). InnerVolumeSpecName "kube-api-access-klw5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:54:55.174177 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.174153    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28699a1d-d60d-4b66-9fe6-1327f52f585e-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:54:55.174278 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.174176    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28699a1d-d60d-4b66-9fe6-1327f52f585e-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:54:55.174278 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.174193    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klw5t\" (UniqueName: \"kubernetes.io/projected/28699a1d-d60d-4b66-9fe6-1327f52f585e-kube-api-access-klw5t\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:54:55.208447 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.208424    2554 generic.go:356] "Generic (PLEG): container finished" podID="28699a1d-d60d-4b66-9fe6-1327f52f585e" containerID="1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1" exitCode=0
Jul 10 13:54:55.208544 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.208470    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"
Jul 10 13:54:55.208544 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.208477    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w" event={"ID":"28699a1d-d60d-4b66-9fe6-1327f52f585e","Type":"ContainerDied","Data":"1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1"}
Jul 10 13:54:55.208619 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.208549    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w" event={"ID":"28699a1d-d60d-4b66-9fe6-1327f52f585e","Type":"ContainerDied","Data":"995cc428dd9c25394b45274b836ed6168c6597facba424917f9a450a53d26c09"}
Jul 10 13:54:55.208619 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.208573    2554 scope.go:117] "RemoveContainer" containerID="1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1"
Jul 10 13:54:55.217690 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.217674    2554 scope.go:117] "RemoveContainer" containerID="1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1"
Jul 10 13:54:55.217951 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:54:55.217931    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1\": container with ID starting with 1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1 not found: ID does not exist" containerID="1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1"
Jul 10 13:54:55.218012 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.217961    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1"} err="failed to get container status \"1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1\": rpc error: code = NotFound desc = could not find container \"1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1\": container with ID starting with 1e5b08437878e33815dd1761d3c009eeb0082698b822855e448d5e1a6130d9a1 not found: ID does not exist"
Jul 10 13:54:55.233908 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.233860    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"]
Jul 10 13:54:55.241445 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:55.241425    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d792a168-59c9975d8f-qwm9w"]
Jul 10 13:54:56.665480 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:54:56.665453    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28699a1d-d60d-4b66-9fe6-1327f52f585e" path="/var/lib/kubelet/pods/28699a1d-d60d-4b66-9fe6-1327f52f585e/volumes"
Jul 10 13:55:28.247116 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.247076    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"]
Jul 10 13:55:28.247666 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.247649    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28699a1d-d60d-4b66-9fe6-1327f52f585e" containerName="maas-api"
Jul 10 13:55:28.247712 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.247669    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="28699a1d-d60d-4b66-9fe6-1327f52f585e" containerName="maas-api"
Jul 10 13:55:28.247764 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.247754    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="28699a1d-d60d-4b66-9fe6-1327f52f585e" containerName="maas-api"
Jul 10 13:55:28.251409 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.251385    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.254753 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.254731    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1a15ffa7-serving-cert\""
Jul 10 13:55:28.267981 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.267958    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"]
Jul 10 13:55:28.331360 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.331332    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.331503 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.331368    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be6e54f9-ce0c-46fe-bf75-37ffa466822a-openshift-service-ca\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.331503 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.331456    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pdjq\" (UniqueName: \"kubernetes.io/projected/be6e54f9-ce0c-46fe-bf75-37ffa466822a-kube-api-access-9pdjq\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.432690 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.432661    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pdjq\" (UniqueName: \"kubernetes.io/projected/be6e54f9-ce0c-46fe-bf75-37ffa466822a-kube-api-access-9pdjq\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.432846 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.432718    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.432846 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.432739    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be6e54f9-ce0c-46fe-bf75-37ffa466822a-openshift-service-ca\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.432846 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:55:28.432812    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-serving-cert: secret "maas-api-e2e-mt-1a15ffa7-serving-cert" not found
Jul 10 13:55:28.432969 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:55:28.432897    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls podName:be6e54f9-ce0c-46fe-bf75-37ffa466822a nodeName:}" failed. No retries permitted until 2026-07-10 13:55:28.932881469 +0000 UTC m=+1752.799008101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls") pod "maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45" (UID: "be6e54f9-ce0c-46fe-bf75-37ffa466822a") : secret "maas-api-e2e-mt-1a15ffa7-serving-cert" not found
Jul 10 13:55:28.433311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.433294    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be6e54f9-ce0c-46fe-bf75-37ffa466822a-openshift-service-ca\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.442172 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.442150    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pdjq\" (UniqueName: \"kubernetes.io/projected/be6e54f9-ce0c-46fe-bf75-37ffa466822a-kube-api-access-9pdjq\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.937659 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.937630    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:28.940059 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:28.940037    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls\") pod \"maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:29.162495 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:29.162459    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:29.300199 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:29.300165    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"]
Jul 10 13:55:29.304606 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:55:29.304580    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe6e54f9_ce0c_46fe_bf75_37ffa466822a.slice/crio-d8395e78a18085f91952f19b49b879b40cf22e8cdd2da644d716de702704c767 WatchSource:0}: Error finding container d8395e78a18085f91952f19b49b879b40cf22e8cdd2da644d716de702704c767: Status 404 returned error can't find the container with id d8395e78a18085f91952f19b49b879b40cf22e8cdd2da644d716de702704c767
Jul 10 13:55:29.366857 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:29.366799    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45" event={"ID":"be6e54f9-ce0c-46fe-bf75-37ffa466822a","Type":"ContainerStarted","Data":"d8395e78a18085f91952f19b49b879b40cf22e8cdd2da644d716de702704c767"}
Jul 10 13:55:30.372220 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:30.372189    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45" event={"ID":"be6e54f9-ce0c-46fe-bf75-37ffa466822a","Type":"ContainerStarted","Data":"f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c"}
Jul 10 13:55:30.372616 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:30.372316    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:30.389968 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:30.389926    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45" podStartSLOduration=1.96385652 podStartE2EDuration="2.389912476s" totalImagesPullingTime="426.055956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:55:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:55:30.388949375 +0000 UTC m=+1754.255076029" watchObservedRunningTime="2026-07-10 13:55:30.389912476 +0000 UTC m=+1754.256039155"
Jul 10 13:55:36.382008 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:36.381977    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:55:46.814670 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.814634    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"]
Jul 10 13:55:46.818337 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.818314    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:46.821008 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.820987    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1a15ffa7\"/\"kube-root-ca.crt\""
Jul 10 13:55:46.822166 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.822148    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1a15ffa7\"/\"openshift-service-ca.crt\""
Jul 10 13:55:46.822244 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.822151    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1a15ffa7\"/\"e2e-lifecycle-model-1a15ffa7-kserve-self-signed-certs\""
Jul 10 13:55:46.822244 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.822192    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1a15ffa7\"/\"default-dockercfg-jgtpr\""
Jul 10 13:55:46.830148 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.830126    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"]
Jul 10 13:55:46.972134 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.972103    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgznv\" (UniqueName: \"kubernetes.io/projected/cc73a6fe-8f4f-431a-a604-6e409b598c94-kube-api-access-pgznv\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:46.972324 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.972186    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-home\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:46.972324 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.972223    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-dshm\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:46.972324 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.972313    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-model-cache\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:46.972481 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.972353    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-tmp-dir\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:46.972481 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:46.972380    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc73a6fe-8f4f-431a-a604-6e409b598c94-tls-certs\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.073799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.073725    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgznv\" (UniqueName: \"kubernetes.io/projected/cc73a6fe-8f4f-431a-a604-6e409b598c94-kube-api-access-pgznv\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.073799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.073764    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-home\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.073799 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.073781    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-dshm\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.074035 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.073841    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-model-cache\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.074035 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.073883    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-tmp-dir\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.074035 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.073902    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc73a6fe-8f4f-431a-a604-6e409b598c94-tls-certs\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.074180 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.074157    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-home\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.074263 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.074241    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-tmp-dir\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.074322 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.074258    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-model-cache\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.076063 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.076042    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-dshm\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.076447 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.076426    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc73a6fe-8f4f-431a-a604-6e409b598c94-tls-certs\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.083116 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.083096    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgznv\" (UniqueName: \"kubernetes.io/projected/cc73a6fe-8f4f-431a-a604-6e409b598c94-kube-api-access-pgznv\") pod \"e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") " pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.132137 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.132111    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:47.266011 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.265986    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"]
Jul 10 13:55:47.268707 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:55:47.268681    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc73a6fe_8f4f_431a_a604_6e409b598c94.slice/crio-4df6d759362663094ed138dc2f4b287c03a72d03672a0e42f75a77271b7f5115 WatchSource:0}: Error finding container 4df6d759362663094ed138dc2f4b287c03a72d03672a0e42f75a77271b7f5115: Status 404 returned error can't find the container with id 4df6d759362663094ed138dc2f4b287c03a72d03672a0e42f75a77271b7f5115
Jul 10 13:55:47.448517 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.448481    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg" event={"ID":"cc73a6fe-8f4f-431a-a604-6e409b598c94","Type":"ContainerStarted","Data":"e3f76d3be7904051391c2a684d08bf1c0886edb0b92ce5db7ea6307634c79ad7"}
Jul 10 13:55:47.448664 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.448523    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg" event={"ID":"cc73a6fe-8f4f-431a-a604-6e409b598c94","Type":"ContainerStarted","Data":"4df6d759362663094ed138dc2f4b287c03a72d03672a0e42f75a77271b7f5115"}
Jul 10 13:55:47.467340 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:47.467284    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg" podStartSLOduration=1.467264404 podStartE2EDuration="1.467264404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:55:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:55:47.465210377 +0000 UTC m=+1771.331337258" watchObservedRunningTime="2026-07-10 13:55:47.467264404 +0000 UTC m=+1771.333391063"
Jul 10 13:55:57.132551 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:57.132515    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:57.133067 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:57.132589    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:57.145046 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:57.145022    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:55:57.510491 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:55:57.510465    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:56:10.705743 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:10.705551    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"]
Jul 10 13:56:10.706371 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:10.706342    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45" podUID="be6e54f9-ce0c-46fe-bf75-37ffa466822a" containerName="maas-api" containerID="cri-o://f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c" gracePeriod=30
Jul 10 13:56:10.957023 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:10.956971    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:56:11.060103 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.060077    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls\") pod \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") "
Jul 10 13:56:11.060278 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.060135    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pdjq\" (UniqueName: \"kubernetes.io/projected/be6e54f9-ce0c-46fe-bf75-37ffa466822a-kube-api-access-9pdjq\") pod \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") "
Jul 10 13:56:11.060278 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.060197    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be6e54f9-ce0c-46fe-bf75-37ffa466822a-openshift-service-ca\") pod \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\" (UID: \"be6e54f9-ce0c-46fe-bf75-37ffa466822a\") "
Jul 10 13:56:11.060597 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.060573    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be6e54f9-ce0c-46fe-bf75-37ffa466822a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be6e54f9-ce0c-46fe-bf75-37ffa466822a" (UID: "be6e54f9-ce0c-46fe-bf75-37ffa466822a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:56:11.062015 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.061987    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be6e54f9-ce0c-46fe-bf75-37ffa466822a" (UID: "be6e54f9-ce0c-46fe-bf75-37ffa466822a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:56:11.062142 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.062126    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be6e54f9-ce0c-46fe-bf75-37ffa466822a-kube-api-access-9pdjq" (OuterVolumeSpecName: "kube-api-access-9pdjq") pod "be6e54f9-ce0c-46fe-bf75-37ffa466822a" (UID: "be6e54f9-ce0c-46fe-bf75-37ffa466822a"). InnerVolumeSpecName "kube-api-access-9pdjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:56:11.160996 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.160967    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be6e54f9-ce0c-46fe-bf75-37ffa466822a-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:11.160996 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.160990    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pdjq\" (UniqueName: \"kubernetes.io/projected/be6e54f9-ce0c-46fe-bf75-37ffa466822a-kube-api-access-9pdjq\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:11.160996 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.161001    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be6e54f9-ce0c-46fe-bf75-37ffa466822a-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:11.566589 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.566559    2554 generic.go:356] "Generic (PLEG): container finished" podID="be6e54f9-ce0c-46fe-bf75-37ffa466822a" containerID="f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c" exitCode=0
Jul 10 13:56:11.566764 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.566620    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45" event={"ID":"be6e54f9-ce0c-46fe-bf75-37ffa466822a","Type":"ContainerDied","Data":"f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c"}
Jul 10 13:56:11.566764 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.566630    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"
Jul 10 13:56:11.566764 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.566663    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45" event={"ID":"be6e54f9-ce0c-46fe-bf75-37ffa466822a","Type":"ContainerDied","Data":"d8395e78a18085f91952f19b49b879b40cf22e8cdd2da644d716de702704c767"}
Jul 10 13:56:11.566764 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.566684    2554 scope.go:117] "RemoveContainer" containerID="f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c"
Jul 10 13:56:11.575586 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.575566    2554 scope.go:117] "RemoveContainer" containerID="f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c"
Jul 10 13:56:11.575878 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:56:11.575858    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c\": container with ID starting with f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c not found: ID does not exist" containerID="f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c"
Jul 10 13:56:11.575966 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.575892    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c"} err="failed to get container status \"f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c\": rpc error: code = NotFound desc = could not find container \"f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c\": container with ID starting with f0bffe3ed3bccf71bd267b01ba30e9d1c525b37269b2836a892dcbe8cc2aa32c not found: ID does not exist"
Jul 10 13:56:11.591363 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.591342    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"]
Jul 10 13:56:11.597912 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:11.597893    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1a15ffa7-77c479d54c-h4b45"]
Jul 10 13:56:12.664968 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:12.664928    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be6e54f9-ce0c-46fe-bf75-37ffa466822a" path="/var/lib/kubelet/pods/be6e54f9-ce0c-46fe-bf75-37ffa466822a/volumes"
Jul 10 13:56:21.480874 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:21.480837    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"]
Jul 10 13:56:36.500741 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.500685    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg" podUID="cc73a6fe-8f4f-431a-a604-6e409b598c94" containerName="main" containerID="cri-o://e3f76d3be7904051391c2a684d08bf1c0886edb0b92ce5db7ea6307634c79ad7" gracePeriod=45
Jul 10 13:56:36.683980 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.683945    2554 generic.go:356] "Generic (PLEG): container finished" podID="cc73a6fe-8f4f-431a-a604-6e409b598c94" containerID="e3f76d3be7904051391c2a684d08bf1c0886edb0b92ce5db7ea6307634c79ad7" exitCode=0
Jul 10 13:56:36.684144 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.684095    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg" event={"ID":"cc73a6fe-8f4f-431a-a604-6e409b598c94","Type":"ContainerDied","Data":"e3f76d3be7904051391c2a684d08bf1c0886edb0b92ce5db7ea6307634c79ad7"}
Jul 10 13:56:36.748528 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.748503    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:56:36.856853 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.856761    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-model-cache\") pod \"cc73a6fe-8f4f-431a-a604-6e409b598c94\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") "
Jul 10 13:56:36.856853 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.856792    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-home\") pod \"cc73a6fe-8f4f-431a-a604-6e409b598c94\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") "
Jul 10 13:56:36.856853 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.856812    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc73a6fe-8f4f-431a-a604-6e409b598c94-tls-certs\") pod \"cc73a6fe-8f4f-431a-a604-6e409b598c94\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") "
Jul 10 13:56:36.856853 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.856850    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgznv\" (UniqueName: \"kubernetes.io/projected/cc73a6fe-8f4f-431a-a604-6e409b598c94-kube-api-access-pgznv\") pod \"cc73a6fe-8f4f-431a-a604-6e409b598c94\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") "
Jul 10 13:56:36.857168 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.856876    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-dshm\") pod \"cc73a6fe-8f4f-431a-a604-6e409b598c94\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") "
Jul 10 13:56:36.857168 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.856963    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-tmp-dir\") pod \"cc73a6fe-8f4f-431a-a604-6e409b598c94\" (UID: \"cc73a6fe-8f4f-431a-a604-6e409b598c94\") "
Jul 10 13:56:36.857168 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.857117    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-home" (OuterVolumeSpecName: "home") pod "cc73a6fe-8f4f-431a-a604-6e409b598c94" (UID: "cc73a6fe-8f4f-431a-a604-6e409b598c94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:56:36.857168 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.857126    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc73a6fe-8f4f-431a-a604-6e409b598c94" (UID: "cc73a6fe-8f4f-431a-a604-6e409b598c94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:56:36.857349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.857242    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc73a6fe-8f4f-431a-a604-6e409b598c94" (UID: "cc73a6fe-8f4f-431a-a604-6e409b598c94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:56:36.857349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.857324    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:36.857349 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.857338    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:36.857447 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.857351    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:36.859055 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.859036    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc73a6fe-8f4f-431a-a604-6e409b598c94-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc73a6fe-8f4f-431a-a604-6e409b598c94" (UID: "cc73a6fe-8f4f-431a-a604-6e409b598c94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:56:36.859118 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.859073    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-dshm" (OuterVolumeSpecName: "dshm") pod "cc73a6fe-8f4f-431a-a604-6e409b598c94" (UID: "cc73a6fe-8f4f-431a-a604-6e409b598c94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:56:36.859155 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.859140    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc73a6fe-8f4f-431a-a604-6e409b598c94-kube-api-access-pgznv" (OuterVolumeSpecName: "kube-api-access-pgznv") pod "cc73a6fe-8f4f-431a-a604-6e409b598c94" (UID: "cc73a6fe-8f4f-431a-a604-6e409b598c94"). InnerVolumeSpecName "kube-api-access-pgznv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:56:36.958241 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.958216    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc73a6fe-8f4f-431a-a604-6e409b598c94-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:36.958241 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.958239    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgznv\" (UniqueName: \"kubernetes.io/projected/cc73a6fe-8f4f-431a-a604-6e409b598c94-kube-api-access-pgznv\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:36.958394 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:36.958248    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc73a6fe-8f4f-431a-a604-6e409b598c94-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:56:37.689274 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:37.689247    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"
Jul 10 13:56:37.689705 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:37.689246    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg" event={"ID":"cc73a6fe-8f4f-431a-a604-6e409b598c94","Type":"ContainerDied","Data":"4df6d759362663094ed138dc2f4b287c03a72d03672a0e42f75a77271b7f5115"}
Jul 10 13:56:37.689705 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:37.689365    2554 scope.go:117] "RemoveContainer" containerID="e3f76d3be7904051391c2a684d08bf1c0886edb0b92ce5db7ea6307634c79ad7"
Jul 10 13:56:37.711652 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:37.711624    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"]
Jul 10 13:56:37.717940 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:37.717912    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-1a15ffa7/e2e-lifecycle-model-1a15ffa7-kserve-6fcc4c555-xmddg"]
Jul 10 13:56:38.664201 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:56:38.664172    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc73a6fe-8f4f-431a-a604-6e409b598c94" path="/var/lib/kubelet/pods/cc73a6fe-8f4f-431a-a604-6e409b598c94/volumes"
Jul 10 13:58:05.918920 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.918883    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"]
Jul 10 13:58:05.919368 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.919251    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc73a6fe-8f4f-431a-a604-6e409b598c94" containerName="main"
Jul 10 13:58:05.919368 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.919263    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc73a6fe-8f4f-431a-a604-6e409b598c94" containerName="main"
Jul 10 13:58:05.919368 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.919277    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be6e54f9-ce0c-46fe-bf75-37ffa466822a" containerName="maas-api"
Jul 10 13:58:05.919368 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.919282    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="be6e54f9-ce0c-46fe-bf75-37ffa466822a" containerName="maas-api"
Jul 10 13:58:05.919368 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.919337    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="be6e54f9-ce0c-46fe-bf75-37ffa466822a" containerName="maas-api"
Jul 10 13:58:05.919368 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.919346    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc73a6fe-8f4f-431a-a604-6e409b598c94" containerName="main"
Jul 10 13:58:05.922375 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.922348    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:05.925024 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.925001    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6c6c27-serving-cert\""
Jul 10 13:58:05.933413 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:05.933391    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"]
Jul 10 13:58:06.032216 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.032183    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.032379 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.032238    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27jzv\" (UniqueName: \"kubernetes.io/projected/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-kube-api-access-27jzv\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.032379 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.032322    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.133160 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.133128    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27jzv\" (UniqueName: \"kubernetes.io/projected/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-kube-api-access-27jzv\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.133327 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.133178    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.133327 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.133219    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.133415 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:58:06.133334    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-serving-cert: secret "maas-api-e2e-inf-a-6c6c27-serving-cert" not found
Jul 10 13:58:06.133415 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:58:06.133399    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls podName:3fb5c22b-175a-42e2-a717-1e477b0b9bf9 nodeName:}" failed. No retries permitted until 2026-07-10 13:58:06.633380966 +0000 UTC m=+1910.499507617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls") pod "maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64" (UID: "3fb5c22b-175a-42e2-a717-1e477b0b9bf9") : secret "maas-api-e2e-inf-a-6c6c27-serving-cert" not found
Jul 10 13:58:06.133810 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.133791    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.150107 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.150079    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27jzv\" (UniqueName: \"kubernetes.io/projected/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-kube-api-access-27jzv\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.637398 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.637361    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.639958 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.639931    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls\") pod \"maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.833908 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.833871    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:06.974551 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.974527    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"]
Jul 10 13:58:06.977695 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:58:06.977646    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fb5c22b_175a_42e2_a717_1e477b0b9bf9.slice/crio-7f51f411f32aa1508c55de652c4935b69d56e5881db03cf2e46523d749a5c1ce WatchSource:0}: Error finding container 7f51f411f32aa1508c55de652c4935b69d56e5881db03cf2e46523d749a5c1ce: Status 404 returned error can't find the container with id 7f51f411f32aa1508c55de652c4935b69d56e5881db03cf2e46523d749a5c1ce
Jul 10 13:58:06.981565 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:06.981544    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 13:58:07.089723 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:07.089686    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64" event={"ID":"3fb5c22b-175a-42e2-a717-1e477b0b9bf9","Type":"ContainerStarted","Data":"7f51f411f32aa1508c55de652c4935b69d56e5881db03cf2e46523d749a5c1ce"}
Jul 10 13:58:08.096462 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:08.096376    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64" event={"ID":"3fb5c22b-175a-42e2-a717-1e477b0b9bf9","Type":"ContainerStarted","Data":"98986b48129047285a95243a5e3714441698471a2462d12426340ea2d8d67c58"}
Jul 10 13:58:08.096853 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:08.096500    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:08.118249 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:08.118198    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64" podStartSLOduration=2.414838144 podStartE2EDuration="3.118183042s" totalImagesPullingTime="703.344898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:58:08.116808823 +0000 UTC m=+1911.982935512" watchObservedRunningTime="2026-07-10 13:58:08.118183042 +0000 UTC m=+1911.984309739"
Jul 10 13:58:14.031556 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.031520    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"]
Jul 10 13:58:14.035016 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.034990    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.038448 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.038428    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5cb1b4-serving-cert\""
Jul 10 13:58:14.048264 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.048242    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"]
Jul 10 13:58:14.106175 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.106152    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:58:14.198227 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.198193    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.198428 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.198254    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj6f9\" (UniqueName: \"kubernetes.io/projected/2f5530ee-c417-4815-8d8c-9d2f191dee3e-kube-api-access-tj6f9\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.198428 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.198280    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5530ee-c417-4815-8d8c-9d2f191dee3e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.299721 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.299644    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.299721 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.299676    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj6f9\" (UniqueName: \"kubernetes.io/projected/2f5530ee-c417-4815-8d8c-9d2f191dee3e-kube-api-access-tj6f9\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.299721 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.299701    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5530ee-c417-4815-8d8c-9d2f191dee3e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.300026 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:58:14.299782    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-serving-cert: secret "maas-api-e2e-inf-b-5cb1b4-serving-cert" not found
Jul 10 13:58:14.300026 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:58:14.299872    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls podName:2f5530ee-c417-4815-8d8c-9d2f191dee3e nodeName:}" failed. No retries permitted until 2026-07-10 13:58:14.799852833 +0000 UTC m=+1918.665979467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls") pod "maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" (UID: "2f5530ee-c417-4815-8d8c-9d2f191dee3e") : secret "maas-api-e2e-inf-b-5cb1b4-serving-cert" not found
Jul 10 13:58:14.300280 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.300263    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5530ee-c417-4815-8d8c-9d2f191dee3e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.309735 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.309712    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj6f9\" (UniqueName: \"kubernetes.io/projected/2f5530ee-c417-4815-8d8c-9d2f191dee3e-kube-api-access-tj6f9\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.803613 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:14.803578    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:14.803779 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:58:14.803723    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-serving-cert: secret "maas-api-e2e-inf-b-5cb1b4-serving-cert" not found
Jul 10 13:58:14.803839 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:58:14.803786    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls podName:2f5530ee-c417-4815-8d8c-9d2f191dee3e nodeName:}" failed. No retries permitted until 2026-07-10 13:58:15.803768838 +0000 UTC m=+1919.669895473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls") pod "maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" (UID: "2f5530ee-c417-4815-8d8c-9d2f191dee3e") : secret "maas-api-e2e-inf-b-5cb1b4-serving-cert" not found
Jul 10 13:58:15.812957 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:15.812920    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:15.815780 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:15.815753    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls\") pod \"maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:15.846455 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:15.846428    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:15.985086 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:15.985059    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"]
Jul 10 13:58:15.986758 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:58:15.986732    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5530ee_c417_4815_8d8c_9d2f191dee3e.slice/crio-fa5db9084add90070ed6fd9165e29ebc189664922d0f445dfbfa31219bcd3a50 WatchSource:0}: Error finding container fa5db9084add90070ed6fd9165e29ebc189664922d0f445dfbfa31219bcd3a50: Status 404 returned error can't find the container with id fa5db9084add90070ed6fd9165e29ebc189664922d0f445dfbfa31219bcd3a50
Jul 10 13:58:16.131745 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:16.131676    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" event={"ID":"2f5530ee-c417-4815-8d8c-9d2f191dee3e","Type":"ContainerStarted","Data":"fa5db9084add90070ed6fd9165e29ebc189664922d0f445dfbfa31219bcd3a50"}
Jul 10 13:58:17.137479 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:17.137434    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" event={"ID":"2f5530ee-c417-4815-8d8c-9d2f191dee3e","Type":"ContainerStarted","Data":"9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1"}
Jul 10 13:58:17.137880 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:17.137519    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:17.156340 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:17.156288    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" podStartSLOduration=2.708811132 podStartE2EDuration="3.156272574s" totalImagesPullingTime="447.461442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:58:17.155109327 +0000 UTC m=+1921.021235982" watchObservedRunningTime="2026-07-10 13:58:17.156272574 +0000 UTC m=+1921.022399228"
Jul 10 13:58:23.147144 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:23.147111    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 13:58:26.708728 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.708698    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"]
Jul 10 13:58:26.712541 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.712515    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.715582 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.715555    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6c6c27\"/\"kube-root-ca.crt\""
Jul 10 13:58:26.716669 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.716648    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6c6c27\"/\"openshift-service-ca.crt\""
Jul 10 13:58:26.716791 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.716693    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6c6c27\"/\"test-model-6c6c27-kserve-self-signed-certs\""
Jul 10 13:58:26.716791 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.716699    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6c6c27\"/\"default-dockercfg-n7gvd\""
Jul 10 13:58:26.722071 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.722050    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"]
Jul 10 13:58:26.794336 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.794309    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-home\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.794473 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.794354    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6ksm\" (UniqueName: \"kubernetes.io/projected/04438624-c94b-4a42-9845-758798731b28-kube-api-access-j6ksm\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.794473 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.794400    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-dshm\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.794565 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.794500    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-model-cache\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.794565 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.794558    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04438624-c94b-4a42-9845-758798731b28-tls-certs\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.794645 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.794576    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-tmp-dir\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.895982 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.895958    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-model-cache\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896113 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896006    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04438624-c94b-4a42-9845-758798731b28-tls-certs\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896113 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896033    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-tmp-dir\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896113 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896076    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-home\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896261 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896209    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6ksm\" (UniqueName: \"kubernetes.io/projected/04438624-c94b-4a42-9845-758798731b28-kube-api-access-j6ksm\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896318 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896261    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-dshm\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896389 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896369    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-model-cache\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896444 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896390    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-tmp-dir\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.896579 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.896559    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-home\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.898388 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.898361    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-dshm\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.898489 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.898473    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04438624-c94b-4a42-9845-758798731b28-tls-certs\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:26.907140 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:26.907112    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6ksm\" (UniqueName: \"kubernetes.io/projected/04438624-c94b-4a42-9845-758798731b28-kube-api-access-j6ksm\") pod \"test-model-6c6c27-kserve-66c566694c-ssgk6\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") " pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:27.021961 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:27.021886    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:27.159614 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:27.159582    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"]
Jul 10 13:58:27.162323 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:58:27.162292    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04438624_c94b_4a42_9845_758798731b28.slice/crio-796b2603d53c03873d233f8f69c40d53a31ab8296e65e1e2ed3c1f47c49a24eb WatchSource:0}: Error finding container 796b2603d53c03873d233f8f69c40d53a31ab8296e65e1e2ed3c1f47c49a24eb: Status 404 returned error can't find the container with id 796b2603d53c03873d233f8f69c40d53a31ab8296e65e1e2ed3c1f47c49a24eb
Jul 10 13:58:27.185494 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:27.185471    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6" event={"ID":"04438624-c94b-4a42-9845-758798731b28","Type":"ContainerStarted","Data":"796b2603d53c03873d233f8f69c40d53a31ab8296e65e1e2ed3c1f47c49a24eb"}
Jul 10 13:58:28.192257 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:28.192195    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6" event={"ID":"04438624-c94b-4a42-9845-758798731b28","Type":"ContainerStarted","Data":"66ba02f087ea60e45e20521dab66be9a37fca809542e86ebb1aba46a2df189f8"}
Jul 10 13:58:28.210060 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:28.210006    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6" podStartSLOduration=2.209992144 podStartE2EDuration="2.209992144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:58:28.208465394 +0000 UTC m=+1932.074592049" watchObservedRunningTime="2026-07-10 13:58:28.209992144 +0000 UTC m=+1932.076118799"
Jul 10 13:58:37.022012 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:37.021983    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:37.022553 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:37.022119    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:37.035683 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:37.035661    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:37.245620 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:37.245595    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:58:56.726789 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.726755    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"]
Jul 10 13:58:56.730322 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.730306    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.734015 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.733993    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5cb1b4\"/\"default-dockercfg-ksl48\""
Jul 10 13:58:56.734151 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.734025    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5cb1b4\"/\"openshift-service-ca.crt\""
Jul 10 13:58:56.734151 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.734030    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5cb1b4\"/\"kube-root-ca.crt\""
Jul 10 13:58:56.734151 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.734091    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5cb1b4\"/\"test-model-5cb1b4-kserve-self-signed-certs\""
Jul 10 13:58:56.745906 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.741165    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"]
Jul 10 13:58:56.850556 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.850523    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-model-cache\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.850556 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.850559    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af6c974-b821-4736-a755-11f6af764e3e-tls-certs\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.850780 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.850582    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-tmp-dir\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.850780 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.850607    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-dshm\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.850780 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.850660    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88wrf\" (UniqueName: \"kubernetes.io/projected/0af6c974-b821-4736-a755-11f6af764e3e-kube-api-access-88wrf\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.850780 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.850726    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-home\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952022 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.951994    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-model-cache\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952189 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952030    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af6c974-b821-4736-a755-11f6af764e3e-tls-certs\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952189 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952055    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-tmp-dir\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952276 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952189    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-dshm\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952322 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952278    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88wrf\" (UniqueName: \"kubernetes.io/projected/0af6c974-b821-4736-a755-11f6af764e3e-kube-api-access-88wrf\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952366 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952342    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-home\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952433 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952412    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-tmp-dir\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952501 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952460    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-model-cache\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.952630 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.952610    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-home\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.954478 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.954453    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-dshm\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.954686 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.954658    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af6c974-b821-4736-a755-11f6af764e3e-tls-certs\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:56.963575 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:56.963549    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88wrf\" (UniqueName: \"kubernetes.io/projected/0af6c974-b821-4736-a755-11f6af764e3e-kube-api-access-88wrf\") pod \"test-model-5cb1b4-kserve-5854c98d99-q8dgk\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") " pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:57.040597 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:57.040530    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:58:57.178218 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:57.178189    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"]
Jul 10 13:58:57.180198 ip-10-0-135-59 kubenswrapper[2554]: W0710 13:58:57.180169    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0af6c974_b821_4736_a755_11f6af764e3e.slice/crio-256e5d0686a27abda985f31a02aa9b7dfc85c78f0d697553cc6a94f694ef48bd WatchSource:0}: Error finding container 256e5d0686a27abda985f31a02aa9b7dfc85c78f0d697553cc6a94f694ef48bd: Status 404 returned error can't find the container with id 256e5d0686a27abda985f31a02aa9b7dfc85c78f0d697553cc6a94f694ef48bd
Jul 10 13:58:57.330942 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:57.330853    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk" event={"ID":"0af6c974-b821-4736-a755-11f6af764e3e","Type":"ContainerStarted","Data":"a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6"}
Jul 10 13:58:57.330942 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:57.330900    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk" event={"ID":"0af6c974-b821-4736-a755-11f6af764e3e","Type":"ContainerStarted","Data":"256e5d0686a27abda985f31a02aa9b7dfc85c78f0d697553cc6a94f694ef48bd"}
Jul 10 13:58:57.350606 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:58:57.350547    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk" podStartSLOduration=1.3505264289999999 podStartE2EDuration="1.350526429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:58:57.349351694 +0000 UTC m=+1961.215478349" watchObservedRunningTime="2026-07-10 13:58:57.350526429 +0000 UTC m=+1961.216653083"
Jul 10 13:59:07.041369 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:07.041337    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:59:07.041757 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:07.041391    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:59:07.053895 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:07.053872    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:59:07.392342 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:07.392264    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:59:17.307835 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:17.307745    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"]
Jul 10 13:59:18.213386 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:18.213351    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"]
Jul 10 13:59:19.207394 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.207352    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"]
Jul 10 13:59:19.207863 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.207577    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64" podUID="3fb5c22b-175a-42e2-a717-1e477b0b9bf9" containerName="maas-api" containerID="cri-o://98986b48129047285a95243a5e3714441698471a2462d12426340ea2d8d67c58" gracePeriod=30
Jul 10 13:59:19.442602 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.442560    2554 generic.go:356] "Generic (PLEG): container finished" podID="3fb5c22b-175a-42e2-a717-1e477b0b9bf9" containerID="98986b48129047285a95243a5e3714441698471a2462d12426340ea2d8d67c58" exitCode=0
Jul 10 13:59:19.442773 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.442617    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64" event={"ID":"3fb5c22b-175a-42e2-a717-1e477b0b9bf9","Type":"ContainerDied","Data":"98986b48129047285a95243a5e3714441698471a2462d12426340ea2d8d67c58"}
Jul 10 13:59:19.466522 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.466466    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:59:19.648870 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.648840    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls\") pod \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") "
Jul 10 13:59:19.649022 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.648950    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27jzv\" (UniqueName: \"kubernetes.io/projected/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-kube-api-access-27jzv\") pod \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") "
Jul 10 13:59:19.649022 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.648987    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-openshift-service-ca\") pod \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\" (UID: \"3fb5c22b-175a-42e2-a717-1e477b0b9bf9\") "
Jul 10 13:59:19.649391 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.649360    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3fb5c22b-175a-42e2-a717-1e477b0b9bf9" (UID: "3fb5c22b-175a-42e2-a717-1e477b0b9bf9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 13:59:19.651079 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.651046    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-kube-api-access-27jzv" (OuterVolumeSpecName: "kube-api-access-27jzv") pod "3fb5c22b-175a-42e2-a717-1e477b0b9bf9" (UID: "3fb5c22b-175a-42e2-a717-1e477b0b9bf9"). InnerVolumeSpecName "kube-api-access-27jzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:59:19.651311 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.651278    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3fb5c22b-175a-42e2-a717-1e477b0b9bf9" (UID: "3fb5c22b-175a-42e2-a717-1e477b0b9bf9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:59:19.749851 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.749764    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:19.749851 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.749790    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:19.749851 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:19.749799    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27jzv\" (UniqueName: \"kubernetes.io/projected/3fb5c22b-175a-42e2-a717-1e477b0b9bf9-kube-api-access-27jzv\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:20.450164 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:20.450135    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"
Jul 10 13:59:20.450584 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:20.450169    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64" event={"ID":"3fb5c22b-175a-42e2-a717-1e477b0b9bf9","Type":"ContainerDied","Data":"7f51f411f32aa1508c55de652c4935b69d56e5881db03cf2e46523d749a5c1ce"}
Jul 10 13:59:20.450584 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:20.450235    2554 scope.go:117] "RemoveContainer" containerID="98986b48129047285a95243a5e3714441698471a2462d12426340ea2d8d67c58"
Jul 10 13:59:20.476209 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:20.476147    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"]
Jul 10 13:59:20.478370 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:20.478337    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6c6c27-68fdf86c56-rll64"]
Jul 10 13:59:20.664637 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:20.664607    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fb5c22b-175a-42e2-a717-1e477b0b9bf9" path="/var/lib/kubelet/pods/3fb5c22b-175a-42e2-a717-1e477b0b9bf9/volumes"
Jul 10 13:59:32.327740 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.327697    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6" podUID="04438624-c94b-4a42-9845-758798731b28" containerName="main" containerID="cri-o://66ba02f087ea60e45e20521dab66be9a37fca809542e86ebb1aba46a2df189f8" gracePeriod=45
Jul 10 13:59:32.510776 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.510749    2554 generic.go:356] "Generic (PLEG): container finished" podID="04438624-c94b-4a42-9845-758798731b28" containerID="66ba02f087ea60e45e20521dab66be9a37fca809542e86ebb1aba46a2df189f8" exitCode=0
Jul 10 13:59:32.510916 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.510839    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6" event={"ID":"04438624-c94b-4a42-9845-758798731b28","Type":"ContainerDied","Data":"66ba02f087ea60e45e20521dab66be9a37fca809542e86ebb1aba46a2df189f8"}
Jul 10 13:59:32.591729 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.591669    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:59:32.653393 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653367    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04438624-c94b-4a42-9845-758798731b28-tls-certs\") pod \"04438624-c94b-4a42-9845-758798731b28\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") "
Jul 10 13:59:32.653521 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653429    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-model-cache\") pod \"04438624-c94b-4a42-9845-758798731b28\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") "
Jul 10 13:59:32.653521 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653460    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-home\") pod \"04438624-c94b-4a42-9845-758798731b28\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") "
Jul 10 13:59:32.653612 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653550    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-tmp-dir\") pod \"04438624-c94b-4a42-9845-758798731b28\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") "
Jul 10 13:59:32.653612 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653588    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6ksm\" (UniqueName: \"kubernetes.io/projected/04438624-c94b-4a42-9845-758798731b28-kube-api-access-j6ksm\") pod \"04438624-c94b-4a42-9845-758798731b28\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") "
Jul 10 13:59:32.653720 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653617    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-dshm\") pod \"04438624-c94b-4a42-9845-758798731b28\" (UID: \"04438624-c94b-4a42-9845-758798731b28\") "
Jul 10 13:59:32.653775 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653713    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-model-cache" (OuterVolumeSpecName: "model-cache") pod "04438624-c94b-4a42-9845-758798731b28" (UID: "04438624-c94b-4a42-9845-758798731b28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:32.653849 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653783    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04438624-c94b-4a42-9845-758798731b28" (UID: "04438624-c94b-4a42-9845-758798731b28"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:32.653849 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653800    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-home" (OuterVolumeSpecName: "home") pod "04438624-c94b-4a42-9845-758798731b28" (UID: "04438624-c94b-4a42-9845-758798731b28"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:32.653960 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653915    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:32.653960 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.653937    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:32.655691 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.655669    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04438624-c94b-4a42-9845-758798731b28-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04438624-c94b-4a42-9845-758798731b28" (UID: "04438624-c94b-4a42-9845-758798731b28"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:59:32.655913 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.655873    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04438624-c94b-4a42-9845-758798731b28-kube-api-access-j6ksm" (OuterVolumeSpecName: "kube-api-access-j6ksm") pod "04438624-c94b-4a42-9845-758798731b28" (UID: "04438624-c94b-4a42-9845-758798731b28"). InnerVolumeSpecName "kube-api-access-j6ksm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:59:32.656222 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.656195    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-dshm" (OuterVolumeSpecName: "dshm") pod "04438624-c94b-4a42-9845-758798731b28" (UID: "04438624-c94b-4a42-9845-758798731b28"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:32.754938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.754908    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:32.754938 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.754935    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6ksm\" (UniqueName: \"kubernetes.io/projected/04438624-c94b-4a42-9845-758798731b28-kube-api-access-j6ksm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:32.755095 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.754949    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04438624-c94b-4a42-9845-758798731b28-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:32.755095 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:32.754963    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04438624-c94b-4a42-9845-758798731b28-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:33.233649 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.233613    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk" podUID="0af6c974-b821-4736-a755-11f6af764e3e" containerName="main" containerID="cri-o://a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6" gracePeriod=45
Jul 10 13:59:33.490067 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.490013    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:59:33.516696 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.516663    2554 generic.go:356] "Generic (PLEG): container finished" podID="0af6c974-b821-4736-a755-11f6af764e3e" containerID="a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6" exitCode=0
Jul 10 13:59:33.516925 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.516882    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk" event={"ID":"0af6c974-b821-4736-a755-11f6af764e3e","Type":"ContainerDied","Data":"a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6"}
Jul 10 13:59:33.517048 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.516941    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk" event={"ID":"0af6c974-b821-4736-a755-11f6af764e3e","Type":"ContainerDied","Data":"256e5d0686a27abda985f31a02aa9b7dfc85c78f0d697553cc6a94f694ef48bd"}
Jul 10 13:59:33.517048 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.516961    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"
Jul 10 13:59:33.517048 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.516970    2554 scope.go:117] "RemoveContainer" containerID="a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6"
Jul 10 13:59:33.518635 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.518612    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6" event={"ID":"04438624-c94b-4a42-9845-758798731b28","Type":"ContainerDied","Data":"796b2603d53c03873d233f8f69c40d53a31ab8296e65e1e2ed3c1f47c49a24eb"}
Jul 10 13:59:33.518719 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.518655    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"
Jul 10 13:59:33.526043 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.526014    2554 scope.go:117] "RemoveContainer" containerID="a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6"
Jul 10 13:59:33.526299 ip-10-0-135-59 kubenswrapper[2554]: E0710 13:59:33.526282    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6\": container with ID starting with a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6 not found: ID does not exist" containerID="a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6"
Jul 10 13:59:33.526361 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.526307    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6"} err="failed to get container status \"a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6\": rpc error: code = NotFound desc = could not find container \"a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6\": container with ID starting with a4a038a282341de69c0b23d4a047f3508ac736c4b0b34edea5b674e03ab140d6 not found: ID does not exist"
Jul 10 13:59:33.526361 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.526323    2554 scope.go:117] "RemoveContainer" containerID="66ba02f087ea60e45e20521dab66be9a37fca809542e86ebb1aba46a2df189f8"
Jul 10 13:59:33.537931 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.537909    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"]
Jul 10 13:59:33.545043 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.545022    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6c6c27/test-model-6c6c27-kserve-66c566694c-ssgk6"]
Jul 10 13:59:33.561692 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.561673    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88wrf\" (UniqueName: \"kubernetes.io/projected/0af6c974-b821-4736-a755-11f6af764e3e-kube-api-access-88wrf\") pod \"0af6c974-b821-4736-a755-11f6af764e3e\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") "
Jul 10 13:59:33.561786 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.561732    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-dshm\") pod \"0af6c974-b821-4736-a755-11f6af764e3e\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") "
Jul 10 13:59:33.561786 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.561766    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-home\") pod \"0af6c974-b821-4736-a755-11f6af764e3e\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") "
Jul 10 13:59:33.561920 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.561801    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-tmp-dir\") pod \"0af6c974-b821-4736-a755-11f6af764e3e\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") "
Jul 10 13:59:33.561920 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.561871    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af6c974-b821-4736-a755-11f6af764e3e-tls-certs\") pod \"0af6c974-b821-4736-a755-11f6af764e3e\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") "
Jul 10 13:59:33.561920 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.561906    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-model-cache\") pod \"0af6c974-b821-4736-a755-11f6af764e3e\" (UID: \"0af6c974-b821-4736-a755-11f6af764e3e\") "
Jul 10 13:59:33.562121 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.562094    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-home" (OuterVolumeSpecName: "home") pod "0af6c974-b821-4736-a755-11f6af764e3e" (UID: "0af6c974-b821-4736-a755-11f6af764e3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:33.562121 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.562112    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0af6c974-b821-4736-a755-11f6af764e3e" (UID: "0af6c974-b821-4736-a755-11f6af764e3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:33.562244 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.562198    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:33.562244 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.562217    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:33.562348 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.562282    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "0af6c974-b821-4736-a755-11f6af764e3e" (UID: "0af6c974-b821-4736-a755-11f6af764e3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:33.564022 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.563996    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-dshm" (OuterVolumeSpecName: "dshm") pod "0af6c974-b821-4736-a755-11f6af764e3e" (UID: "0af6c974-b821-4736-a755-11f6af764e3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 13:59:33.564119 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.564064    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af6c974-b821-4736-a755-11f6af764e3e-kube-api-access-88wrf" (OuterVolumeSpecName: "kube-api-access-88wrf") pod "0af6c974-b821-4736-a755-11f6af764e3e" (UID: "0af6c974-b821-4736-a755-11f6af764e3e"). InnerVolumeSpecName "kube-api-access-88wrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 13:59:33.564186 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.564153    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af6c974-b821-4736-a755-11f6af764e3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0af6c974-b821-4736-a755-11f6af764e3e" (UID: "0af6c974-b821-4736-a755-11f6af764e3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 13:59:33.663000 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.662975    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:33.663000 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.662999    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88wrf\" (UniqueName: \"kubernetes.io/projected/0af6c974-b821-4736-a755-11f6af764e3e-kube-api-access-88wrf\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:33.663134 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.663013    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0af6c974-b821-4736-a755-11f6af764e3e-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:33.663134 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.663028    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0af6c974-b821-4736-a755-11f6af764e3e-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 13:59:33.840635 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.840611    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"]
Jul 10 13:59:33.845357 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:33.845337    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5cb1b4/test-model-5cb1b4-kserve-5854c98d99-q8dgk"]
Jul 10 13:59:34.668638 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:34.668604    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04438624-c94b-4a42-9845-758798731b28" path="/var/lib/kubelet/pods/04438624-c94b-4a42-9845-758798731b28/volumes"
Jul 10 13:59:34.669038 ip-10-0-135-59 kubenswrapper[2554]: I0710 13:59:34.668990    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0af6c974-b821-4736-a755-11f6af764e3e" path="/var/lib/kubelet/pods/0af6c974-b821-4736-a755-11f6af764e3e/volumes"
Jul 10 14:00:00.145388 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145356    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"]
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145715    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04438624-c94b-4a42-9845-758798731b28" containerName="main"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145727    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="04438624-c94b-4a42-9845-758798731b28" containerName="main"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145745    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0af6c974-b821-4736-a755-11f6af764e3e" containerName="main"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145750    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0af6c974-b821-4736-a755-11f6af764e3e" containerName="main"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145759    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fb5c22b-175a-42e2-a717-1e477b0b9bf9" containerName="maas-api"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145765    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fb5c22b-175a-42e2-a717-1e477b0b9bf9" containerName="maas-api"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145812    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="04438624-c94b-4a42-9845-758798731b28" containerName="main"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145833    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0af6c974-b821-4736-a755-11f6af764e3e" containerName="main"
Jul 10 14:00:00.147703 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.145843    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fb5c22b-175a-42e2-a717-1e477b0b9bf9" containerName="maas-api"
Jul 10 14:00:00.148448 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.148434    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"]
Jul 10 14:00:00.148572 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.148555    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"
Jul 10 14:00:00.151150 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.151128    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g69k2\""
Jul 10 14:00:00.151452 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.151437    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"
Jul 10 14:00:00.156446 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.156404    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"]
Jul 10 14:00:00.159017 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.158997    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"]
Jul 10 14:00:00.161172 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.161151    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"
Jul 10 14:00:00.168110 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.168091    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"
Jul 10 14:00:00.516731 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.510649    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"]
Jul 10 14:00:00.518654 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:00:00.518623    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe4c107_9156_4cdf_99d9_935320f8956a.slice/crio-ae28ebebb69482524bb2e9bed3a5a3580cc4171342e0f9cf6cd519a9d5402494 WatchSource:0}: Error finding container ae28ebebb69482524bb2e9bed3a5a3580cc4171342e0f9cf6cd519a9d5402494: Status 404 returned error can't find the container with id ae28ebebb69482524bb2e9bed3a5a3580cc4171342e0f9cf6cd519a9d5402494
Jul 10 14:00:00.539282 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.539253    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"]
Jul 10 14:00:00.540652 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:00:00.540630    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53650c7_3ade_4d21_80b5_dc13b83f009b.slice/crio-d180dcd774a93646883c0de7060048203bb8b4aad9dd6bedef00507e2c8f1850 WatchSource:0}: Error finding container d180dcd774a93646883c0de7060048203bb8b4aad9dd6bedef00507e2c8f1850: Status 404 returned error can't find the container with id d180dcd774a93646883c0de7060048203bb8b4aad9dd6bedef00507e2c8f1850
Jul 10 14:00:00.643648 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.643615    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t" event={"ID":"c53650c7-3ade-4d21-80b5-dc13b83f009b","Type":"ContainerStarted","Data":"d180dcd774a93646883c0de7060048203bb8b4aad9dd6bedef00507e2c8f1850"}
Jul 10 14:00:00.644784 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:00.644760    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj" event={"ID":"ebe4c107-9156-4cdf-99d9-935320f8956a","Type":"ContainerStarted","Data":"ae28ebebb69482524bb2e9bed3a5a3580cc4171342e0f9cf6cd519a9d5402494"}
Jul 10 14:00:01.650557 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:01.650517    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t" event={"ID":"c53650c7-3ade-4d21-80b5-dc13b83f009b","Type":"ContainerStarted","Data":"5f381609fd60f5605f65a66575db7460a6c5090e82c6d6b2d4306d0403ec2798"}
Jul 10 14:00:01.652045 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:01.652016    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj" event={"ID":"ebe4c107-9156-4cdf-99d9-935320f8956a","Type":"ContainerStarted","Data":"e507c046fb91d065b06310cf9c2b7cfe91635e63831ecd2d77493e497c6a323e"}
Jul 10 14:00:01.668437 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:01.668392    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t" podStartSLOduration=1.668378846 podStartE2EDuration="1.668378846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:00:01.66714205 +0000 UTC m=+2025.533268705" watchObservedRunningTime="2026-07-10 14:00:01.668378846 +0000 UTC m=+2025.534505550"
Jul 10 14:00:01.683965 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:01.683926    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj" podStartSLOduration=1.683912314 podStartE2EDuration="1.683912314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:00:01.683544585 +0000 UTC m=+2025.549671239" watchObservedRunningTime="2026-07-10 14:00:01.683912314 +0000 UTC m=+2025.550038967"
Jul 10 14:00:03.035167 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.035136    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"]
Jul 10 14:00:03.035644 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.035356    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" podUID="2f5530ee-c417-4815-8d8c-9d2f191dee3e" containerName="maas-api" containerID="cri-o://9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1" gracePeriod=30
Jul 10 14:00:03.070497 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.070473    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"]
Jul 10 14:00:03.143082 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.143041    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" podUID="2f5530ee-c417-4815-8d8c-9d2f191dee3e" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.56:8443/health\": dial tcp 10.134.0.56:8443: connect: connection refused"
Jul 10 14:00:03.289727 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.289674    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 14:00:03.398786 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.398758    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5530ee-c417-4815-8d8c-9d2f191dee3e-openshift-service-ca\") pod \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") "
Jul 10 14:00:03.398981 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.398800    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj6f9\" (UniqueName: \"kubernetes.io/projected/2f5530ee-c417-4815-8d8c-9d2f191dee3e-kube-api-access-tj6f9\") pod \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") "
Jul 10 14:00:03.398981 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.398846    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls\") pod \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\" (UID: \"2f5530ee-c417-4815-8d8c-9d2f191dee3e\") "
Jul 10 14:00:03.399180 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.399158    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f5530ee-c417-4815-8d8c-9d2f191dee3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f5530ee-c417-4815-8d8c-9d2f191dee3e" (UID: "2f5530ee-c417-4815-8d8c-9d2f191dee3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:00:03.400869 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.400841    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f5530ee-c417-4815-8d8c-9d2f191dee3e" (UID: "2f5530ee-c417-4815-8d8c-9d2f191dee3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:00:03.400959 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.400881    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f5530ee-c417-4815-8d8c-9d2f191dee3e-kube-api-access-tj6f9" (OuterVolumeSpecName: "kube-api-access-tj6f9") pod "2f5530ee-c417-4815-8d8c-9d2f191dee3e" (UID: "2f5530ee-c417-4815-8d8c-9d2f191dee3e"). InnerVolumeSpecName "kube-api-access-tj6f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:00:03.500082 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.500057    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj6f9\" (UniqueName: \"kubernetes.io/projected/2f5530ee-c417-4815-8d8c-9d2f191dee3e-kube-api-access-tj6f9\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:03.500082 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.500080    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5530ee-c417-4815-8d8c-9d2f191dee3e-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:03.500207 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.500090    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5530ee-c417-4815-8d8c-9d2f191dee3e-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:03.663483 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.663396    2554 generic.go:356] "Generic (PLEG): container finished" podID="2f5530ee-c417-4815-8d8c-9d2f191dee3e" containerID="9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1" exitCode=0
Jul 10 14:00:03.663483 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.663448    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"
Jul 10 14:00:03.663652 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.663478    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" event={"ID":"2f5530ee-c417-4815-8d8c-9d2f191dee3e","Type":"ContainerDied","Data":"9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1"}
Jul 10 14:00:03.663652 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.663521    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng" event={"ID":"2f5530ee-c417-4815-8d8c-9d2f191dee3e","Type":"ContainerDied","Data":"fa5db9084add90070ed6fd9165e29ebc189664922d0f445dfbfa31219bcd3a50"}
Jul 10 14:00:03.663652 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.663542    2554 scope.go:117] "RemoveContainer" containerID="9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1"
Jul 10 14:00:03.663813 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.663689    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t" podUID="c53650c7-3ade-4d21-80b5-dc13b83f009b" containerName="cleanup" containerID="cri-o://5f381609fd60f5605f65a66575db7460a6c5090e82c6d6b2d4306d0403ec2798" gracePeriod=30
Jul 10 14:00:03.672318 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.672303    2554 scope.go:117] "RemoveContainer" containerID="9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1"
Jul 10 14:00:03.672538 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:00:03.672523    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1\": container with ID starting with 9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1 not found: ID does not exist" containerID="9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1"
Jul 10 14:00:03.672597 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.672546    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1"} err="failed to get container status \"9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1\": rpc error: code = NotFound desc = could not find container \"9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1\": container with ID starting with 9b5a2f4911313aabb731df2d0123e76896f66a9dc3d1cd23481222c63ab69ec1 not found: ID does not exist"
Jul 10 14:00:03.696164 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.696144    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"]
Jul 10 14:00:03.701212 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:03.701191    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5cb1b4-6cbd97789c-5pgng"]
Jul 10 14:00:04.664284 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:04.664247    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f5530ee-c417-4815-8d8c-9d2f191dee3e" path="/var/lib/kubelet/pods/2f5530ee-c417-4815-8d8c-9d2f191dee3e/volumes"
Jul 10 14:00:13.588899 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.588859    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"]
Jul 10 14:00:13.589520 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.589496    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f5530ee-c417-4815-8d8c-9d2f191dee3e" containerName="maas-api"
Jul 10 14:00:13.589520 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.589520    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f5530ee-c417-4815-8d8c-9d2f191dee3e" containerName="maas-api"
Jul 10 14:00:13.589724 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.589617    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f5530ee-c417-4815-8d8c-9d2f191dee3e" containerName="maas-api"
Jul 10 14:00:13.593853 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.593834    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.596410 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.596393    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1667f3-serving-cert\""
Jul 10 14:00:13.602254 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.602231    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"]
Jul 10 14:00:13.677083 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.677056    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-openshift-service-ca\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.677239 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.677116    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-maas-api-tls\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.677239 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.677177    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcxnw\" (UniqueName: \"kubernetes.io/projected/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-kube-api-access-rcxnw\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.778086 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.778054    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxnw\" (UniqueName: \"kubernetes.io/projected/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-kube-api-access-rcxnw\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.778235 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.778097    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-openshift-service-ca\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.778235 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.778143    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-maas-api-tls\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.778808 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.778788    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-openshift-service-ca\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.780541 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.780519    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-maas-api-tls\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.787308 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.787286    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcxnw\" (UniqueName: \"kubernetes.io/projected/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-kube-api-access-rcxnw\") pod \"maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:13.905227 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:13.905171    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:14.041692 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:14.041621    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"]
Jul 10 14:00:14.043853 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:00:14.043799    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3c30d4_fdd3_4397_8b8b_1830af5fc095.slice/crio-d0694b5b41d65bca66dad1f298b2f675b6ee78808ce38455e72fac8ab3bcba3d WatchSource:0}: Error finding container d0694b5b41d65bca66dad1f298b2f675b6ee78808ce38455e72fac8ab3bcba3d: Status 404 returned error can't find the container with id d0694b5b41d65bca66dad1f298b2f675b6ee78808ce38455e72fac8ab3bcba3d
Jul 10 14:00:14.713540 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:14.713505    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc" event={"ID":"3f3c30d4-fdd3-4397-8b8b-1830af5fc095","Type":"ContainerStarted","Data":"d0694b5b41d65bca66dad1f298b2f675b6ee78808ce38455e72fac8ab3bcba3d"}
Jul 10 14:00:15.719899 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:15.719862    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc" event={"ID":"3f3c30d4-fdd3-4397-8b8b-1830af5fc095","Type":"ContainerStarted","Data":"40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c"}
Jul 10 14:00:15.720280 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:15.719925    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:15.741643 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:15.741600    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc" podStartSLOduration=2.038981636 podStartE2EDuration="2.741587063s" totalImagesPullingTime="702.605427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:00:15.740361716 +0000 UTC m=+2039.606488375" watchObservedRunningTime="2026-07-10 14:00:15.741587063 +0000 UTC m=+2039.607713717"
Jul 10 14:00:21.625295 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.625265    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"]
Jul 10 14:00:21.628790 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.628766    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.631339 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.631311    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5da7f6-serving-cert\""
Jul 10 14:00:21.639756 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.639730    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"]
Jul 10 14:00:21.730088 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.730056    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:21.745847 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.745814    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pwj5\" (UniqueName: \"kubernetes.io/projected/35c8c8f6-ef67-402e-a375-a1e70abeb58d-kube-api-access-8pwj5\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.745986 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.745915    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.745986 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.745962    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c8c8f6-ef67-402e-a375-a1e70abeb58d-openshift-service-ca\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.847070 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.847043    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.847239 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.847075    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c8c8f6-ef67-402e-a375-a1e70abeb58d-openshift-service-ca\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.847239 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.847110    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pwj5\" (UniqueName: \"kubernetes.io/projected/35c8c8f6-ef67-402e-a375-a1e70abeb58d-kube-api-access-8pwj5\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.847239 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:00:21.847184    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-serving-cert: secret "maas-api-e2e-api-b-5da7f6-serving-cert" not found
Jul 10 14:00:21.847424 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:00:21.847249    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls podName:35c8c8f6-ef67-402e-a375-a1e70abeb58d nodeName:}" failed. No retries permitted until 2026-07-10 14:00:22.347229621 +0000 UTC m=+2046.213356266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls") pod "maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn" (UID: "35c8c8f6-ef67-402e-a375-a1e70abeb58d") : secret "maas-api-e2e-api-b-5da7f6-serving-cert" not found
Jul 10 14:00:21.847758 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.847734    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c8c8f6-ef67-402e-a375-a1e70abeb58d-openshift-service-ca\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:21.860542 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:21.860510    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pwj5\" (UniqueName: \"kubernetes.io/projected/35c8c8f6-ef67-402e-a375-a1e70abeb58d-kube-api-access-8pwj5\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:22.351998 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:22.351964    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:22.354617 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:22.354580    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls\") pod \"maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:22.541120 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:22.541090    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:22.684650 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:22.684611    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"]
Jul 10 14:00:22.686604 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:00:22.686577    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c8c8f6_ef67_402e_a375_a1e70abeb58d.slice/crio-131b7c982ff74d172d6200c456ea0a9c998a582c3019346c5e5fcadf7dd755b1 WatchSource:0}: Error finding container 131b7c982ff74d172d6200c456ea0a9c998a582c3019346c5e5fcadf7dd755b1: Status 404 returned error can't find the container with id 131b7c982ff74d172d6200c456ea0a9c998a582c3019346c5e5fcadf7dd755b1
Jul 10 14:00:22.753154 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:22.753118    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn" event={"ID":"35c8c8f6-ef67-402e-a375-a1e70abeb58d","Type":"ContainerStarted","Data":"131b7c982ff74d172d6200c456ea0a9c998a582c3019346c5e5fcadf7dd755b1"}
Jul 10 14:00:23.760662 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:23.760622    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn" event={"ID":"35c8c8f6-ef67-402e-a375-a1e70abeb58d","Type":"ContainerStarted","Data":"0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3"}
Jul 10 14:00:23.761047 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:23.760768    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:23.779522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:23.779474    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn" podStartSLOduration=2.248027002 podStartE2EDuration="2.779458897s" totalImagesPullingTime="531.431895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:00:23.778472694 +0000 UTC m=+2047.644599351" watchObservedRunningTime="2026-07-10 14:00:23.779458897 +0000 UTC m=+2047.645585550"
Jul 10 14:00:29.769679 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:29.769646    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:33.810048 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:33.810010    2554 generic.go:356] "Generic (PLEG): container finished" podID="c53650c7-3ade-4d21-80b5-dc13b83f009b" containerID="5f381609fd60f5605f65a66575db7460a6c5090e82c6d6b2d4306d0403ec2798" exitCode=137
Jul 10 14:00:33.810403 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:33.810066    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t" event={"ID":"c53650c7-3ade-4d21-80b5-dc13b83f009b","Type":"ContainerDied","Data":"5f381609fd60f5605f65a66575db7460a6c5090e82c6d6b2d4306d0403ec2798"}
Jul 10 14:00:33.858071 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:33.858049    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"
Jul 10 14:00:34.816022 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:34.815983    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t" event={"ID":"c53650c7-3ade-4d21-80b5-dc13b83f009b","Type":"ContainerDied","Data":"d180dcd774a93646883c0de7060048203bb8b4aad9dd6bedef00507e2c8f1850"}
Jul 10 14:00:34.816022 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:34.816023    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"
Jul 10 14:00:34.816501 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:34.816078    2554 scope.go:117] "RemoveContainer" containerID="5f381609fd60f5605f65a66575db7460a6c5090e82c6d6b2d4306d0403ec2798"
Jul 10 14:00:34.837396 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:34.837372    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"]
Jul 10 14:00:34.846472 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:34.846450    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5cb1b4-29728200-mkw9t"]
Jul 10 14:00:35.086122 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.086049    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"]
Jul 10 14:00:35.086340 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.086284    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc" podUID="3f3c30d4-fdd3-4397-8b8b-1830af5fc095" containerName="maas-api" containerID="cri-o://40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c" gracePeriod=30
Jul 10 14:00:35.327201 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.327178    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:35.456381 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.456347    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-openshift-service-ca\") pod \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") "
Jul 10 14:00:35.456563 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.456457    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-maas-api-tls\") pod \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") "
Jul 10 14:00:35.456563 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.456505    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcxnw\" (UniqueName: \"kubernetes.io/projected/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-kube-api-access-rcxnw\") pod \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\" (UID: \"3f3c30d4-fdd3-4397-8b8b-1830af5fc095\") "
Jul 10 14:00:35.456769 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.456736    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3f3c30d4-fdd3-4397-8b8b-1830af5fc095" (UID: "3f3c30d4-fdd3-4397-8b8b-1830af5fc095"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:00:35.458617 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.458585    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-kube-api-access-rcxnw" (OuterVolumeSpecName: "kube-api-access-rcxnw") pod "3f3c30d4-fdd3-4397-8b8b-1830af5fc095" (UID: "3f3c30d4-fdd3-4397-8b8b-1830af5fc095"). InnerVolumeSpecName "kube-api-access-rcxnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:00:35.458741 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.458721    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3f3c30d4-fdd3-4397-8b8b-1830af5fc095" (UID: "3f3c30d4-fdd3-4397-8b8b-1830af5fc095"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:00:35.557372 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.557346    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:35.557372 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.557371    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:35.557546 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.557380    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcxnw\" (UniqueName: \"kubernetes.io/projected/3f3c30d4-fdd3-4397-8b8b-1830af5fc095-kube-api-access-rcxnw\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:35.822522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.822443    2554 generic.go:356] "Generic (PLEG): container finished" podID="3f3c30d4-fdd3-4397-8b8b-1830af5fc095" containerID="40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c" exitCode=0
Jul 10 14:00:35.822522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.822485    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"
Jul 10 14:00:35.823020 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.822521    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc" event={"ID":"3f3c30d4-fdd3-4397-8b8b-1830af5fc095","Type":"ContainerDied","Data":"40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c"}
Jul 10 14:00:35.823020 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.822577    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc" event={"ID":"3f3c30d4-fdd3-4397-8b8b-1830af5fc095","Type":"ContainerDied","Data":"d0694b5b41d65bca66dad1f298b2f675b6ee78808ce38455e72fac8ab3bcba3d"}
Jul 10 14:00:35.823020 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.822608    2554 scope.go:117] "RemoveContainer" containerID="40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c"
Jul 10 14:00:35.831623 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.831608    2554 scope.go:117] "RemoveContainer" containerID="40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c"
Jul 10 14:00:35.831888 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:00:35.831868    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c\": container with ID starting with 40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c not found: ID does not exist" containerID="40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c"
Jul 10 14:00:35.831940 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.831898    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c"} err="failed to get container status \"40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c\": rpc error: code = NotFound desc = could not find container \"40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c\": container with ID starting with 40103bc93628dfea4768cbd2830abe27eaa223c193da8899b3519eac2dfd9a4c not found: ID does not exist"
Jul 10 14:00:35.852240 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.852216    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"]
Jul 10 14:00:35.854919 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:35.854901    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1667f3-849c6fcc75-9g2rc"]
Jul 10 14:00:36.665937 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:36.665906    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f3c30d4-fdd3-4397-8b8b-1830af5fc095" path="/var/lib/kubelet/pods/3f3c30d4-fdd3-4397-8b8b-1830af5fc095/volumes"
Jul 10 14:00:36.666176 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:36.666163    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c53650c7-3ade-4d21-80b5-dc13b83f009b" path="/var/lib/kubelet/pods/c53650c7-3ade-4d21-80b5-dc13b83f009b/volumes"
Jul 10 14:00:43.338132 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.338055    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"]
Jul 10 14:00:43.338583 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.338272    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn" podUID="35c8c8f6-ef67-402e-a375-a1e70abeb58d" containerName="maas-api" containerID="cri-o://0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3" gracePeriod=30
Jul 10 14:00:43.595566 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.595512    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:43.725159 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.725129    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pwj5\" (UniqueName: \"kubernetes.io/projected/35c8c8f6-ef67-402e-a375-a1e70abeb58d-kube-api-access-8pwj5\") pod \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") "
Jul 10 14:00:43.725337 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.725199    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c8c8f6-ef67-402e-a375-a1e70abeb58d-openshift-service-ca\") pod \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") "
Jul 10 14:00:43.725337 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.725272    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls\") pod \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\" (UID: \"35c8c8f6-ef67-402e-a375-a1e70abeb58d\") "
Jul 10 14:00:43.725558 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.725533    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c8c8f6-ef67-402e-a375-a1e70abeb58d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35c8c8f6-ef67-402e-a375-a1e70abeb58d" (UID: "35c8c8f6-ef67-402e-a375-a1e70abeb58d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:00:43.727196 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.727171    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35c8c8f6-ef67-402e-a375-a1e70abeb58d" (UID: "35c8c8f6-ef67-402e-a375-a1e70abeb58d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:00:43.727302 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.727231    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c8c8f6-ef67-402e-a375-a1e70abeb58d-kube-api-access-8pwj5" (OuterVolumeSpecName: "kube-api-access-8pwj5") pod "35c8c8f6-ef67-402e-a375-a1e70abeb58d" (UID: "35c8c8f6-ef67-402e-a375-a1e70abeb58d"). InnerVolumeSpecName "kube-api-access-8pwj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:00:43.826752 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.826724    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pwj5\" (UniqueName: \"kubernetes.io/projected/35c8c8f6-ef67-402e-a375-a1e70abeb58d-kube-api-access-8pwj5\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:43.826752 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.826749    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35c8c8f6-ef67-402e-a375-a1e70abeb58d-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:43.826927 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.826759    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35c8c8f6-ef67-402e-a375-a1e70abeb58d-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:00:43.864588 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.864516    2554 generic.go:356] "Generic (PLEG): container finished" podID="35c8c8f6-ef67-402e-a375-a1e70abeb58d" containerID="0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3" exitCode=0
Jul 10 14:00:43.864588 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.864565    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"
Jul 10 14:00:43.864758 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.864590    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn" event={"ID":"35c8c8f6-ef67-402e-a375-a1e70abeb58d","Type":"ContainerDied","Data":"0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3"}
Jul 10 14:00:43.864758 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.864644    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn" event={"ID":"35c8c8f6-ef67-402e-a375-a1e70abeb58d","Type":"ContainerDied","Data":"131b7c982ff74d172d6200c456ea0a9c998a582c3019346c5e5fcadf7dd755b1"}
Jul 10 14:00:43.864758 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.864674    2554 scope.go:117] "RemoveContainer" containerID="0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3"
Jul 10 14:00:43.874023 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.874005    2554 scope.go:117] "RemoveContainer" containerID="0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3"
Jul 10 14:00:43.874290 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:00:43.874271    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3\": container with ID starting with 0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3 not found: ID does not exist" containerID="0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3"
Jul 10 14:00:43.874335 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.874298    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3"} err="failed to get container status \"0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3\": rpc error: code = NotFound desc = could not find container \"0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3\": container with ID starting with 0b8a6a5b6d77534d69b035f3cca8bb5a5c02d63ab58e2d54c43100780ca585c3 not found: ID does not exist"
Jul 10 14:00:43.889487 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.889461    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"]
Jul 10 14:00:43.896393 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:43.896373    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5da7f6-d9648d976-rfdmn"]
Jul 10 14:00:44.665613 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:44.665581    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c8c8f6-ef67-402e-a375-a1e70abeb58d" path="/var/lib/kubelet/pods/35c8c8f6-ef67-402e-a375-a1e70abeb58d/volumes"
Jul 10 14:00:53.829984 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.829951    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"]
Jul 10 14:00:53.830398 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830381    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f3c30d4-fdd3-4397-8b8b-1830af5fc095" containerName="maas-api"
Jul 10 14:00:53.830441 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830401    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f3c30d4-fdd3-4397-8b8b-1830af5fc095" containerName="maas-api"
Jul 10 14:00:53.830441 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830410    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c8c8f6-ef67-402e-a375-a1e70abeb58d" containerName="maas-api"
Jul 10 14:00:53.830441 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830415    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c8c8f6-ef67-402e-a375-a1e70abeb58d" containerName="maas-api"
Jul 10 14:00:53.830441 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830428    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53650c7-3ade-4d21-80b5-dc13b83f009b" containerName="cleanup"
Jul 10 14:00:53.830441 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830434    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53650c7-3ade-4d21-80b5-dc13b83f009b" containerName="cleanup"
Jul 10 14:00:53.830644 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830484    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f3c30d4-fdd3-4397-8b8b-1830af5fc095" containerName="maas-api"
Jul 10 14:00:53.830644 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830496    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c8c8f6-ef67-402e-a375-a1e70abeb58d" containerName="maas-api"
Jul 10 14:00:53.830644 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.830503    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c53650c7-3ade-4d21-80b5-dc13b83f009b" containerName="cleanup"
Jul 10 14:00:53.833391 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.833368    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:53.836100 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.836076    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-ffeadb-serving-cert\""
Jul 10 14:00:53.842460 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.842433    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"]
Jul 10 14:00:53.897195 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.897170    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txcvr\" (UniqueName: \"kubernetes.io/projected/d393706e-ad8c-4888-ac2a-0c3879cfffcf-kube-api-access-txcvr\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:53.897329 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.897214    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:53.897329 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.897243    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d393706e-ad8c-4888-ac2a-0c3879cfffcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:53.997793 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.997768    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txcvr\" (UniqueName: \"kubernetes.io/projected/d393706e-ad8c-4888-ac2a-0c3879cfffcf-kube-api-access-txcvr\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:53.997950 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.997806    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:53.997950 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.997853    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d393706e-ad8c-4888-ac2a-0c3879cfffcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:53.998034 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:00:53.997980    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-serving-cert: secret "maas-api-e2e-shared-a-ffeadb-serving-cert" not found
Jul 10 14:00:53.998112 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:00:53.998098    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls podName:d393706e-ad8c-4888-ac2a-0c3879cfffcf nodeName:}" failed. No retries permitted until 2026-07-10 14:00:54.498077025 +0000 UTC m=+2078.364203667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls") pod "maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56" (UID: "d393706e-ad8c-4888-ac2a-0c3879cfffcf") : secret "maas-api-e2e-shared-a-ffeadb-serving-cert" not found
Jul 10 14:00:53.998401 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:53.998385    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d393706e-ad8c-4888-ac2a-0c3879cfffcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:54.009369 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:54.009341    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txcvr\" (UniqueName: \"kubernetes.io/projected/d393706e-ad8c-4888-ac2a-0c3879cfffcf-kube-api-access-txcvr\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:54.500998 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:54.500967    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:54.503324 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:54.503302    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:54.745842 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:54.745790    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:54.914258 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:54.914222    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"]
Jul 10 14:00:55.923967 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:55.923933    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56" event={"ID":"d393706e-ad8c-4888-ac2a-0c3879cfffcf","Type":"ContainerStarted","Data":"418480d46348238788c97b20586dbbb73054250eb5aa876e6a8131adaa1df20c"}
Jul 10 14:00:56.931303 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:56.931269    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56" event={"ID":"d393706e-ad8c-4888-ac2a-0c3879cfffcf","Type":"ContainerStarted","Data":"ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc"}
Jul 10 14:00:56.931681 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:56.931330    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:00:56.951620 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:00:56.951571    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56" podStartSLOduration=2.6306478650000003 podStartE2EDuration="3.95155666s" totalImagesPullingTime="1.320908795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:00:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:00:56.950528709 +0000 UTC m=+2080.816655355" watchObservedRunningTime="2026-07-10 14:00:56.95155666 +0000 UTC m=+2080.817683314"
Jul 10 14:01:00.013122 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:00.013084    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"]
Jul 10 14:01:00.013477 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:00.013289    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj" podUID="ebe4c107-9156-4cdf-99d9-935320f8956a" containerName="cleanup" containerID="cri-o://e507c046fb91d065b06310cf9c2b7cfe91635e63831ecd2d77493e497c6a323e" gracePeriod=30
Jul 10 14:01:01.910927 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:01.910893    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"]
Jul 10 14:01:01.915031 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:01.915004    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:01.917925 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:01.917899    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-c21824-serving-cert\""
Jul 10 14:01:01.927369 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:01.927345    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"]
Jul 10 14:01:01.960237 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:01.960204    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491d9207-8f95-46ff-899a-459eb3ac6d84-maas-api-tls\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:01.960399 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:01.960245    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfbzj\" (UniqueName: \"kubernetes.io/projected/491d9207-8f95-46ff-899a-459eb3ac6d84-kube-api-access-dfbzj\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:01.960399 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:01.960283    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491d9207-8f95-46ff-899a-459eb3ac6d84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.061593 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.061556    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491d9207-8f95-46ff-899a-459eb3ac6d84-maas-api-tls\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.061593 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.061599    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfbzj\" (UniqueName: \"kubernetes.io/projected/491d9207-8f95-46ff-899a-459eb3ac6d84-kube-api-access-dfbzj\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.061796 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.061645    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491d9207-8f95-46ff-899a-459eb3ac6d84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.062367 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.062334    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491d9207-8f95-46ff-899a-459eb3ac6d84-openshift-service-ca\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.064085 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.064056    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491d9207-8f95-46ff-899a-459eb3ac6d84-maas-api-tls\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.071916 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.071889    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfbzj\" (UniqueName: \"kubernetes.io/projected/491d9207-8f95-46ff-899a-459eb3ac6d84-kube-api-access-dfbzj\") pod \"maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.228647 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.228616    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.366025 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.365997    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"]
Jul 10 14:01:02.368290 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:01:02.368258    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491d9207_8f95_46ff_899a_459eb3ac6d84.slice/crio-83c39eaf85060a605460a148cb56fc6b3a6f3bd4f6f67e6081b7c52a2febf532 WatchSource:0}: Error finding container 83c39eaf85060a605460a148cb56fc6b3a6f3bd4f6f67e6081b7c52a2febf532: Status 404 returned error can't find the container with id 83c39eaf85060a605460a148cb56fc6b3a6f3bd4f6f67e6081b7c52a2febf532
Jul 10 14:01:02.941942 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.941912    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:01:02.963026 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.962986    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds" event={"ID":"491d9207-8f95-46ff-899a-459eb3ac6d84","Type":"ContainerStarted","Data":"c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41"}
Jul 10 14:01:02.963195 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.963047    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds" event={"ID":"491d9207-8f95-46ff-899a-459eb3ac6d84","Type":"ContainerStarted","Data":"83c39eaf85060a605460a148cb56fc6b3a6f3bd4f6f67e6081b7c52a2febf532"}
Jul 10 14:01:02.963195 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.963077    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:02.983587 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:02.983529    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds" podStartSLOduration=1.596165869 podStartE2EDuration="1.983508634s" totalImagesPullingTime="387.342765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:01:02.981190072 +0000 UTC m=+2086.847316727" watchObservedRunningTime="2026-07-10 14:01:02.983508634 +0000 UTC m=+2086.849635289"
Jul 10 14:01:08.975925 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:08.975890    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:01:19.313705 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.313674    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"]
Jul 10 14:01:19.318440 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.318417    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.321446 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.321425    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"kube-root-ca.crt\""
Jul 10 14:01:19.322506 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.322482    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"openshift-service-ca.crt\""
Jul 10 14:01:19.322747 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.322726    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"default-dockercfg-b78sx\""
Jul 10 14:01:19.323721 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.323562    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"auth-test-model-ffeadb-kserve-self-signed-certs\""
Jul 10 14:01:19.326289 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.326078    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"]
Jul 10 14:01:19.399348 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.399321    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-dshm\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.399522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.399353    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-home\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.399522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.399381    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-model-cache\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.399522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.399404    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-tmp-dir\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.399522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.399486    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfxrc\" (UniqueName: \"kubernetes.io/projected/3d326e51-f761-47ae-ab21-c9e46ae356c3-kube-api-access-wfxrc\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.399701 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.399523    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d326e51-f761-47ae-ab21-c9e46ae356c3-tls-certs\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500117 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500090    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfxrc\" (UniqueName: \"kubernetes.io/projected/3d326e51-f761-47ae-ab21-c9e46ae356c3-kube-api-access-wfxrc\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500259 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500124    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d326e51-f761-47ae-ab21-c9e46ae356c3-tls-certs\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500259 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500180    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-dshm\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500259 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500201    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-home\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500259 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500233    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-model-cache\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500464 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500260    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-tmp-dir\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500665 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500643    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-home\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500726 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500661    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-model-cache\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.500726 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.500693    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-tmp-dir\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.502466 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.502449    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-dshm\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.502559 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.502544    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d326e51-f761-47ae-ab21-c9e46ae356c3-tls-certs\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.509645 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.509624    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfxrc\" (UniqueName: \"kubernetes.io/projected/3d326e51-f761-47ae-ab21-c9e46ae356c3-kube-api-access-wfxrc\") pod \"auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") " pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.629258 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.629198    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:19.763610 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:19.763573    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"]
Jul 10 14:01:19.765642 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:01:19.765612    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d326e51_f761_47ae_ab21_c9e46ae356c3.slice/crio-5fd7dd472f7e633bcd506e4829c2378907091ecde73cfc5e7268034f3af1c985 WatchSource:0}: Error finding container 5fd7dd472f7e633bcd506e4829c2378907091ecde73cfc5e7268034f3af1c985: Status 404 returned error can't find the container with id 5fd7dd472f7e633bcd506e4829c2378907091ecde73cfc5e7268034f3af1c985
Jul 10 14:01:20.050856 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:20.050801    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz" event={"ID":"3d326e51-f761-47ae-ab21-c9e46ae356c3","Type":"ContainerStarted","Data":"944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214"}
Jul 10 14:01:20.051081 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:20.050867    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz" event={"ID":"3d326e51-f761-47ae-ab21-c9e46ae356c3","Type":"ContainerStarted","Data":"5fd7dd472f7e633bcd506e4829c2378907091ecde73cfc5e7268034f3af1c985"}
Jul 10 14:01:20.070640 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:20.070592    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz" podStartSLOduration=1.070577165 podStartE2EDuration="1.070577165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:01:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:01:20.068636831 +0000 UTC m=+2103.934763497" watchObservedRunningTime="2026-07-10 14:01:20.070577165 +0000 UTC m=+2103.936703818"
Jul 10 14:01:29.629696 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:29.629659    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:29.630090 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:29.629953    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:29.642049 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:29.642024    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:30.100019 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:30.099979    2554 generic.go:356] "Generic (PLEG): container finished" podID="ebe4c107-9156-4cdf-99d9-935320f8956a" containerID="e507c046fb91d065b06310cf9c2b7cfe91635e63831ecd2d77493e497c6a323e" exitCode=137
Jul 10 14:01:30.100142 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:30.100048    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj" event={"ID":"ebe4c107-9156-4cdf-99d9-935320f8956a","Type":"ContainerDied","Data":"e507c046fb91d065b06310cf9c2b7cfe91635e63831ecd2d77493e497c6a323e"}
Jul 10 14:01:30.112068 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:30.112041    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:01:30.203670 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:30.203646    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"
Jul 10 14:01:31.106134 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:31.106111    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"
Jul 10 14:01:31.106134 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:31.106126    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj" event={"ID":"ebe4c107-9156-4cdf-99d9-935320f8956a","Type":"ContainerDied","Data":"ae28ebebb69482524bb2e9bed3a5a3580cc4171342e0f9cf6cd519a9d5402494"}
Jul 10 14:01:31.106621 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:31.106186    2554 scope.go:117] "RemoveContainer" containerID="e507c046fb91d065b06310cf9c2b7cfe91635e63831ecd2d77493e497c6a323e"
Jul 10 14:01:31.124847 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:31.124796    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"]
Jul 10 14:01:31.127013 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:31.126990    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728200-rkgsj"]
Jul 10 14:01:32.664244 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:32.664209    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebe4c107-9156-4cdf-99d9-935320f8956a" path="/var/lib/kubelet/pods/ebe4c107-9156-4cdf-99d9-935320f8956a/volumes"
Jul 10 14:01:58.224577 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.224545    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"]
Jul 10 14:01:58.225043 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.224927    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe4c107-9156-4cdf-99d9-935320f8956a" containerName="cleanup"
Jul 10 14:01:58.225043 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.224938    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe4c107-9156-4cdf-99d9-935320f8956a" containerName="cleanup"
Jul 10 14:01:58.225043 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.225010    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe4c107-9156-4cdf-99d9-935320f8956a" containerName="cleanup"
Jul 10 14:01:58.228228 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.228205    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.230898 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.230877    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"default-dockercfg-gjvr7\""
Jul 10 14:01:58.231014 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.230909    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"kube-root-ca.crt\""
Jul 10 14:01:58.231909 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.231889    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"auth-test-model-c21824-kserve-self-signed-certs\""
Jul 10 14:01:58.232027 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.231926    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"openshift-service-ca.crt\""
Jul 10 14:01:58.237357 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.237337    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"]
Jul 10 14:01:58.407250 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.407215    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-dshm\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.407411 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.407274    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-tmp-dir\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.407411 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.407321    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49130fd-f6a8-4131-86f5-202c72536cc2-tls-certs\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.407411 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.407337    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-home\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.407411 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.407393    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-model-cache\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.407591 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.407428    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vljv8\" (UniqueName: \"kubernetes.io/projected/d49130fd-f6a8-4131-86f5-202c72536cc2-kube-api-access-vljv8\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508345 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508274    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49130fd-f6a8-4131-86f5-202c72536cc2-tls-certs\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508345 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508308    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-home\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508345 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508323    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-model-cache\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508345 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508339    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljv8\" (UniqueName: \"kubernetes.io/projected/d49130fd-f6a8-4131-86f5-202c72536cc2-kube-api-access-vljv8\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508573 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508382    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-dshm\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508573 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508438    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-tmp-dir\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508741 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508705    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-home\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508887 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508750    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-model-cache\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.508887 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.508772    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-tmp-dir\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.510797 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.510770    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-dshm\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.511097 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.511077    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49130fd-f6a8-4131-86f5-202c72536cc2-tls-certs\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.517532 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.517509    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljv8\" (UniqueName: \"kubernetes.io/projected/d49130fd-f6a8-4131-86f5-202c72536cc2-kube-api-access-vljv8\") pod \"auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") " pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.539537 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.539515    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:01:58.677963 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:58.677936    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"]
Jul 10 14:01:58.680153 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:01:58.680125    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd49130fd_f6a8_4131_86f5_202c72536cc2.slice/crio-5a6da98ed51afc1632d904d222619423a0b16af40e2767dfc73e768df99ddf03 WatchSource:0}: Error finding container 5a6da98ed51afc1632d904d222619423a0b16af40e2767dfc73e768df99ddf03: Status 404 returned error can't find the container with id 5a6da98ed51afc1632d904d222619423a0b16af40e2767dfc73e768df99ddf03
Jul 10 14:01:59.237099 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:59.237068    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk" event={"ID":"d49130fd-f6a8-4131-86f5-202c72536cc2","Type":"ContainerStarted","Data":"7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf"}
Jul 10 14:01:59.237460 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:59.237106    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk" event={"ID":"d49130fd-f6a8-4131-86f5-202c72536cc2","Type":"ContainerStarted","Data":"5a6da98ed51afc1632d904d222619423a0b16af40e2767dfc73e768df99ddf03"}
Jul 10 14:01:59.256144 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:01:59.256093    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk" podStartSLOduration=1.256078043 podStartE2EDuration="1.256078043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:01:59.255476951 +0000 UTC m=+2143.121603602" watchObservedRunningTime="2026-07-10 14:01:59.256078043 +0000 UTC m=+2143.122204697"
Jul 10 14:02:08.539922 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:02:08.539838    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:02:08.539922 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:02:08.539891    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:02:08.552382 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:02:08.552356    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:02:09.295137 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:02:09.295106    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:04:11.240100 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:11.240070    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"]
Jul 10 14:04:12.142005 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:12.141961    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"]
Jul 10 14:04:26.261556 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.261502    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz" podUID="3d326e51-f761-47ae-ab21-c9e46ae356c3" containerName="main" containerID="cri-o://944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214" gracePeriod=45
Jul 10 14:04:26.508785 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.508759    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:04:26.578348 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578292    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfxrc\" (UniqueName: \"kubernetes.io/projected/3d326e51-f761-47ae-ab21-c9e46ae356c3-kube-api-access-wfxrc\") pod \"3d326e51-f761-47ae-ab21-c9e46ae356c3\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") "
Jul 10 14:04:26.578348 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578335    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-dshm\") pod \"3d326e51-f761-47ae-ab21-c9e46ae356c3\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") "
Jul 10 14:04:26.578498 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578364    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d326e51-f761-47ae-ab21-c9e46ae356c3-tls-certs\") pod \"3d326e51-f761-47ae-ab21-c9e46ae356c3\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") "
Jul 10 14:04:26.578498 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578398    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-model-cache\") pod \"3d326e51-f761-47ae-ab21-c9e46ae356c3\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") "
Jul 10 14:04:26.578498 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578454    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-tmp-dir\") pod \"3d326e51-f761-47ae-ab21-c9e46ae356c3\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") "
Jul 10 14:04:26.578498 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578480    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-home\") pod \"3d326e51-f761-47ae-ab21-c9e46ae356c3\" (UID: \"3d326e51-f761-47ae-ab21-c9e46ae356c3\") "
Jul 10 14:04:26.578722 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578695    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d326e51-f761-47ae-ab21-c9e46ae356c3" (UID: "3d326e51-f761-47ae-ab21-c9e46ae356c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:26.578791 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578740    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d326e51-f761-47ae-ab21-c9e46ae356c3" (UID: "3d326e51-f761-47ae-ab21-c9e46ae356c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:26.578879 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.578847    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-home" (OuterVolumeSpecName: "home") pod "3d326e51-f761-47ae-ab21-c9e46ae356c3" (UID: "3d326e51-f761-47ae-ab21-c9e46ae356c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:26.580542 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.580519    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-dshm" (OuterVolumeSpecName: "dshm") pod "3d326e51-f761-47ae-ab21-c9e46ae356c3" (UID: "3d326e51-f761-47ae-ab21-c9e46ae356c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:26.580631 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.580520    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d326e51-f761-47ae-ab21-c9e46ae356c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d326e51-f761-47ae-ab21-c9e46ae356c3" (UID: "3d326e51-f761-47ae-ab21-c9e46ae356c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:04:26.580631 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.580622    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d326e51-f761-47ae-ab21-c9e46ae356c3-kube-api-access-wfxrc" (OuterVolumeSpecName: "kube-api-access-wfxrc") pod "3d326e51-f761-47ae-ab21-c9e46ae356c3" (UID: "3d326e51-f761-47ae-ab21-c9e46ae356c3"). InnerVolumeSpecName "kube-api-access-wfxrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:04:26.680034 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.680010    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:26.680034 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.680032    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d326e51-f761-47ae-ab21-c9e46ae356c3-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:26.680166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.680042    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:26.680166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.680049    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:26.680166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.680057    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d326e51-f761-47ae-ab21-c9e46ae356c3-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:26.680166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.680064    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfxrc\" (UniqueName: \"kubernetes.io/projected/3d326e51-f761-47ae-ab21-c9e46ae356c3-kube-api-access-wfxrc\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:26.935140 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.935111    2554 generic.go:356] "Generic (PLEG): container finished" podID="3d326e51-f761-47ae-ab21-c9e46ae356c3" containerID="944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214" exitCode=0
Jul 10 14:04:26.935272 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.935157    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"
Jul 10 14:04:26.935272 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.935198    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz" event={"ID":"3d326e51-f761-47ae-ab21-c9e46ae356c3","Type":"ContainerDied","Data":"944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214"}
Jul 10 14:04:26.935272 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.935238    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz" event={"ID":"3d326e51-f761-47ae-ab21-c9e46ae356c3","Type":"ContainerDied","Data":"5fd7dd472f7e633bcd506e4829c2378907091ecde73cfc5e7268034f3af1c985"}
Jul 10 14:04:26.935272 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.935254    2554 scope.go:117] "RemoveContainer" containerID="944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214"
Jul 10 14:04:26.946166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.943755    2554 scope.go:117] "RemoveContainer" containerID="944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214"
Jul 10 14:04:26.946312 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:04:26.946246    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214\": container with ID starting with 944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214 not found: ID does not exist" containerID="944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214"
Jul 10 14:04:26.946312 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.946278    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214"} err="failed to get container status \"944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214\": rpc error: code = NotFound desc = could not find container \"944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214\": container with ID starting with 944371c292e75e97fa92c3e18363f49c8e5ff355a38fb700f6ad73483d3d4214 not found: ID does not exist"
Jul 10 14:04:26.959560 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.959539    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"]
Jul 10 14:04:26.962176 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:26.962156    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/auth-test-model-ffeadb-kserve-5bb69777dc-4jhwz"]
Jul 10 14:04:27.162722 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.162669    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk" podUID="d49130fd-f6a8-4131-86f5-202c72536cc2" containerName="main" containerID="cri-o://7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf" gracePeriod=45
Jul 10 14:04:27.415203 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.415176    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:04:27.419027 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.419004    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"]
Jul 10 14:04:27.419369 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.419357    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d49130fd-f6a8-4131-86f5-202c72536cc2" containerName="main"
Jul 10 14:04:27.419432 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.419370    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d49130fd-f6a8-4131-86f5-202c72536cc2" containerName="main"
Jul 10 14:04:27.419432 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.419384    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d326e51-f761-47ae-ab21-c9e46ae356c3" containerName="main"
Jul 10 14:04:27.419432 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.419390    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d326e51-f761-47ae-ab21-c9e46ae356c3" containerName="main"
Jul 10 14:04:27.419551 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.419479    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d326e51-f761-47ae-ab21-c9e46ae356c3" containerName="main"
Jul 10 14:04:27.419551 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.419489    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d49130fd-f6a8-4131-86f5-202c72536cc2" containerName="main"
Jul 10 14:04:27.422378 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.422359    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.424886 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.424857    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"openshift-service-ca.crt\""
Jul 10 14:04:27.424886 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.424884    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"default-dockercfg-b78sx\""
Jul 10 14:04:27.425152 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.425131    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"sub-test-model-ffeadb-kserve-self-signed-certs\""
Jul 10 14:04:27.425892 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.425874    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"kube-root-ca.crt\""
Jul 10 14:04:27.434020 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.433994    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"]
Jul 10 14:04:27.486517 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486495    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-home\") pod \"d49130fd-f6a8-4131-86f5-202c72536cc2\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") "
Jul 10 14:04:27.486639 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486529    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49130fd-f6a8-4131-86f5-202c72536cc2-tls-certs\") pod \"d49130fd-f6a8-4131-86f5-202c72536cc2\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") "
Jul 10 14:04:27.486639 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486562    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-dshm\") pod \"d49130fd-f6a8-4131-86f5-202c72536cc2\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") "
Jul 10 14:04:27.486639 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486615    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-model-cache\") pod \"d49130fd-f6a8-4131-86f5-202c72536cc2\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") "
Jul 10 14:04:27.486639 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486635    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-tmp-dir\") pod \"d49130fd-f6a8-4131-86f5-202c72536cc2\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") "
Jul 10 14:04:27.486800 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486657    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vljv8\" (UniqueName: \"kubernetes.io/projected/d49130fd-f6a8-4131-86f5-202c72536cc2-kube-api-access-vljv8\") pod \"d49130fd-f6a8-4131-86f5-202c72536cc2\" (UID: \"d49130fd-f6a8-4131-86f5-202c72536cc2\") "
Jul 10 14:04:27.486800 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486753    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tmp-dir\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.486943 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486806    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-home" (OuterVolumeSpecName: "home") pod "d49130fd-f6a8-4131-86f5-202c72536cc2" (UID: "d49130fd-f6a8-4131-86f5-202c72536cc2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:27.486943 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486872    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-model-cache\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.486943 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486901    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-model-cache" (OuterVolumeSpecName: "model-cache") pod "d49130fd-f6a8-4131-86f5-202c72536cc2" (UID: "d49130fd-f6a8-4131-86f5-202c72536cc2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:27.486943 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486928    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d49130fd-f6a8-4131-86f5-202c72536cc2" (UID: "d49130fd-f6a8-4131-86f5-202c72536cc2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:27.487157 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.486910    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-dshm\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.487157 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.487010    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8g8m\" (UniqueName: \"kubernetes.io/projected/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-kube-api-access-h8g8m\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.487157 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.487042    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tls-certs\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.487157 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.487067    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-home\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.487368 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.487184    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:27.487368 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.487203    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:27.487368 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.487217    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:27.488732 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.488710    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-dshm" (OuterVolumeSpecName: "dshm") pod "d49130fd-f6a8-4131-86f5-202c72536cc2" (UID: "d49130fd-f6a8-4131-86f5-202c72536cc2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:04:27.488815 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.488771    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d49130fd-f6a8-4131-86f5-202c72536cc2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d49130fd-f6a8-4131-86f5-202c72536cc2" (UID: "d49130fd-f6a8-4131-86f5-202c72536cc2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:04:27.488815 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.488781    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49130fd-f6a8-4131-86f5-202c72536cc2-kube-api-access-vljv8" (OuterVolumeSpecName: "kube-api-access-vljv8") pod "d49130fd-f6a8-4131-86f5-202c72536cc2" (UID: "d49130fd-f6a8-4131-86f5-202c72536cc2"). InnerVolumeSpecName "kube-api-access-vljv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:04:27.588553 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588491    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tmp-dir\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.588553 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588534    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-model-cache\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.588696 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588555    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-dshm\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.588696 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588590    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8g8m\" (UniqueName: \"kubernetes.io/projected/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-kube-api-access-h8g8m\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.588696 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588619    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tls-certs\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.588696 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588647    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-home\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.588922 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588709    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49130fd-f6a8-4131-86f5-202c72536cc2-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:27.588922 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588723    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49130fd-f6a8-4131-86f5-202c72536cc2-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:27.588922 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588735    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vljv8\" (UniqueName: \"kubernetes.io/projected/d49130fd-f6a8-4131-86f5-202c72536cc2-kube-api-access-vljv8\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:04:27.588922 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588917    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-model-cache\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.589070 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.588977    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tmp-dir\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.589070 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.589001    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-home\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.590661 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.590634    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-dshm\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.590974 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.590955    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tls-certs\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.597959 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.597934    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8g8m\" (UniqueName: \"kubernetes.io/projected/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-kube-api-access-h8g8m\") pod \"sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") " pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.732098 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.732059    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:27.866841 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.866802    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"]
Jul 10 14:04:27.869167 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:04:27.869139    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bdd2f68_af6b_4db6_9d6c_0909ee11023a.slice/crio-fada783b3d241d9e63af87e9a61bc512a8f27f6250a49c29024bd459624ab5b0 WatchSource:0}: Error finding container fada783b3d241d9e63af87e9a61bc512a8f27f6250a49c29024bd459624ab5b0: Status 404 returned error can't find the container with id fada783b3d241d9e63af87e9a61bc512a8f27f6250a49c29024bd459624ab5b0
Jul 10 14:04:27.942953 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.942926    2554 generic.go:356] "Generic (PLEG): container finished" podID="d49130fd-f6a8-4131-86f5-202c72536cc2" containerID="7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf" exitCode=0
Jul 10 14:04:27.943059 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.942969    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"
Jul 10 14:04:27.943059 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.943008    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk" event={"ID":"d49130fd-f6a8-4131-86f5-202c72536cc2","Type":"ContainerDied","Data":"7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf"}
Jul 10 14:04:27.943059 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.943053    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk" event={"ID":"d49130fd-f6a8-4131-86f5-202c72536cc2","Type":"ContainerDied","Data":"5a6da98ed51afc1632d904d222619423a0b16af40e2767dfc73e768df99ddf03"}
Jul 10 14:04:27.943240 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.943075    2554 scope.go:117] "RemoveContainer" containerID="7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf"
Jul 10 14:04:27.946405 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.946359    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz" event={"ID":"8bdd2f68-af6b-4db6-9d6c-0909ee11023a","Type":"ContainerStarted","Data":"dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f"}
Jul 10 14:04:27.946405 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.946389    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz" event={"ID":"8bdd2f68-af6b-4db6-9d6c-0909ee11023a","Type":"ContainerStarted","Data":"fada783b3d241d9e63af87e9a61bc512a8f27f6250a49c29024bd459624ab5b0"}
Jul 10 14:04:27.952634 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.952616    2554 scope.go:117] "RemoveContainer" containerID="7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf"
Jul 10 14:04:27.952967 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:04:27.952934    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf\": container with ID starting with 7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf not found: ID does not exist" containerID="7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf"
Jul 10 14:04:27.953157 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.952975    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf"} err="failed to get container status \"7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf\": rpc error: code = NotFound desc = could not find container \"7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf\": container with ID starting with 7f4b9b4eb355f220edf2dda018fe797872168477e1e54789f3069eac5d89beaf not found: ID does not exist"
Jul 10 14:04:27.969128 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.969091    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz" podStartSLOduration=0.969077234 podStartE2EDuration="969.077234ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:04:27.967006891 +0000 UTC m=+2291.833133547" watchObservedRunningTime="2026-07-10 14:04:27.969077234 +0000 UTC m=+2291.835203887"
Jul 10 14:04:27.983682 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.983658    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"]
Jul 10 14:04:27.988753 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:27.988730    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/auth-test-model-c21824-kserve-5bb6ccb7bc-vnvnk"]
Jul 10 14:04:28.664625 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:28.664595    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d326e51-f761-47ae-ab21-c9e46ae356c3" path="/var/lib/kubelet/pods/3d326e51-f761-47ae-ab21-c9e46ae356c3/volumes"
Jul 10 14:04:28.665020 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:28.665005    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d49130fd-f6a8-4131-86f5-202c72536cc2" path="/var/lib/kubelet/pods/d49130fd-f6a8-4131-86f5-202c72536cc2/volumes"
Jul 10 14:04:37.732909 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:37.732863    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:37.733287 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:37.732960    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:37.745247 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:37.745220    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:38.004909 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:38.004833    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:04:41.718103 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.718069    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"]
Jul 10 14:04:41.722778 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.722755    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.726535 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.726516    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"default-dockercfg-gjvr7\""
Jul 10 14:04:41.726535 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.726529    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"openshift-service-ca.crt\""
Jul 10 14:04:41.726716 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.726517    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"kube-root-ca.crt\""
Jul 10 14:04:41.726716 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.726567    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"sub-test-model-c21824-kserve-self-signed-certs\""
Jul 10 14:04:41.731567 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.731546    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"]
Jul 10 14:04:41.802384 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.802353    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-model-cache\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.802384 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.802384    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-home\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.802536 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.802417    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lk6p\" (UniqueName: \"kubernetes.io/projected/d571e7c1-d96c-4432-9665-032d0adb8371-kube-api-access-4lk6p\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.802536 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.802504    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-tmp-dir\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.802609 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.802546    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-dshm\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.802609 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.802575    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d571e7c1-d96c-4432-9665-032d0adb8371-tls-certs\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.902955 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.902927    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-model-cache\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.902955 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.902955    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-home\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.903110 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.902986    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lk6p\" (UniqueName: \"kubernetes.io/projected/d571e7c1-d96c-4432-9665-032d0adb8371-kube-api-access-4lk6p\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.903151 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.903113    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-tmp-dir\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.903195 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.903162    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-dshm\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.903232 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.903201    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d571e7c1-d96c-4432-9665-032d0adb8371-tls-certs\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.903317 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.903297    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-model-cache\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.903391 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.903356    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-home\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.903469 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.903450    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-tmp-dir\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.905427 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.905407    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-dshm\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.905594 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.905577    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d571e7c1-d96c-4432-9665-032d0adb8371-tls-certs\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:41.912529 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:41.912496    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lk6p\" (UniqueName: \"kubernetes.io/projected/d571e7c1-d96c-4432-9665-032d0adb8371-kube-api-access-4lk6p\") pod \"sub-test-model-c21824-kserve-8596c84c95-q59xw\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") " pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:42.032515 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:42.032450    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:42.175309 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:42.175284    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"]
Jul 10 14:04:42.177164 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:04:42.177137    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd571e7c1_d96c_4432_9665_032d0adb8371.slice/crio-0637ffde2bd18c476d031ffe7693e6c1630e0920270b1e0d105f0bca51f52245 WatchSource:0}: Error finding container 0637ffde2bd18c476d031ffe7693e6c1630e0920270b1e0d105f0bca51f52245: Status 404 returned error can't find the container with id 0637ffde2bd18c476d031ffe7693e6c1630e0920270b1e0d105f0bca51f52245
Jul 10 14:04:43.018535 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:43.018498    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw" event={"ID":"d571e7c1-d96c-4432-9665-032d0adb8371","Type":"ContainerStarted","Data":"5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5"}
Jul 10 14:04:43.019024 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:43.018546    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw" event={"ID":"d571e7c1-d96c-4432-9665-032d0adb8371","Type":"ContainerStarted","Data":"0637ffde2bd18c476d031ffe7693e6c1630e0920270b1e0d105f0bca51f52245"}
Jul 10 14:04:43.039752 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:43.039703    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw" podStartSLOduration=2.039690287 podStartE2EDuration="2.039690287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:04:43.03739994 +0000 UTC m=+2306.903526593" watchObservedRunningTime="2026-07-10 14:04:43.039690287 +0000 UTC m=+2306.905816940"
Jul 10 14:04:52.032809 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:52.032774    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:52.033204 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:52.032856    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:52.045036 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:52.045013    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:04:52.071974 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:04:52.071946    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:05:58.011017 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:05:58.010986    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"]
Jul 10 14:05:58.821986 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:05:58.821942    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"]
Jul 10 14:06:13.030554 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.030516    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz" podUID="8bdd2f68-af6b-4db6-9d6c-0909ee11023a" containerName="main" containerID="cri-o://dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f" gracePeriod=45
Jul 10 14:06:13.284135 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.284070    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:06:13.447310 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447283    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tls-certs\") pod \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") "
Jul 10 14:06:13.447475 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447346    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-home\") pod \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") "
Jul 10 14:06:13.447475 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447425    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-dshm\") pod \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") "
Jul 10 14:06:13.447475 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447467    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-model-cache\") pod \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") "
Jul 10 14:06:13.447647 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447500    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tmp-dir\") pod \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") "
Jul 10 14:06:13.447647 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447533    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8g8m\" (UniqueName: \"kubernetes.io/projected/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-kube-api-access-h8g8m\") pod \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\" (UID: \"8bdd2f68-af6b-4db6-9d6c-0909ee11023a\") "
Jul 10 14:06:13.447647 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447609    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-home" (OuterVolumeSpecName: "home") pod "8bdd2f68-af6b-4db6-9d6c-0909ee11023a" (UID: "8bdd2f68-af6b-4db6-9d6c-0909ee11023a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:13.447805 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447741    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8bdd2f68-af6b-4db6-9d6c-0909ee11023a" (UID: "8bdd2f68-af6b-4db6-9d6c-0909ee11023a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:13.447805 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447767    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:13.447805 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.447758    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-model-cache" (OuterVolumeSpecName: "model-cache") pod "8bdd2f68-af6b-4db6-9d6c-0909ee11023a" (UID: "8bdd2f68-af6b-4db6-9d6c-0909ee11023a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:13.449730 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.449697    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-dshm" (OuterVolumeSpecName: "dshm") pod "8bdd2f68-af6b-4db6-9d6c-0909ee11023a" (UID: "8bdd2f68-af6b-4db6-9d6c-0909ee11023a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:13.449730 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.449706    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-kube-api-access-h8g8m" (OuterVolumeSpecName: "kube-api-access-h8g8m") pod "8bdd2f68-af6b-4db6-9d6c-0909ee11023a" (UID: "8bdd2f68-af6b-4db6-9d6c-0909ee11023a"). InnerVolumeSpecName "kube-api-access-h8g8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:06:13.449730 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.449725    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8bdd2f68-af6b-4db6-9d6c-0909ee11023a" (UID: "8bdd2f68-af6b-4db6-9d6c-0909ee11023a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:06:13.450171 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.450145    2554 generic.go:356] "Generic (PLEG): container finished" podID="8bdd2f68-af6b-4db6-9d6c-0909ee11023a" containerID="dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f" exitCode=0
Jul 10 14:06:13.450248 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.450190    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"
Jul 10 14:06:13.450248 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.450195    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz" event={"ID":"8bdd2f68-af6b-4db6-9d6c-0909ee11023a","Type":"ContainerDied","Data":"dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f"}
Jul 10 14:06:13.450248 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.450235    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz" event={"ID":"8bdd2f68-af6b-4db6-9d6c-0909ee11023a","Type":"ContainerDied","Data":"fada783b3d241d9e63af87e9a61bc512a8f27f6250a49c29024bd459624ab5b0"}
Jul 10 14:06:13.450397 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.450257    2554 scope.go:117] "RemoveContainer" containerID="dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f"
Jul 10 14:06:13.468299 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.468280    2554 scope.go:117] "RemoveContainer" containerID="dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f"
Jul 10 14:06:13.468543 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:06:13.468524    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f\": container with ID starting with dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f not found: ID does not exist" containerID="dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f"
Jul 10 14:06:13.468611 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.468555    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f"} err="failed to get container status \"dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f\": rpc error: code = NotFound desc = could not find container \"dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f\": container with ID starting with dde9f04967d31595ba2a4bf148a3a7d33dec844c5358eadc845fe31bc21a962f not found: ID does not exist"
Jul 10 14:06:13.481977 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.481956    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"]
Jul 10 14:06:13.484364 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.484343    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/sub-test-model-ffeadb-kserve-76b9c4f867-kp2mz"]
Jul 10 14:06:13.548604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.548553    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:13.548604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.548571    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:13.548604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.548579    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:13.548604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.548588    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8g8m\" (UniqueName: \"kubernetes.io/projected/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-kube-api-access-h8g8m\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:13.548604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.548598    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bdd2f68-af6b-4db6-9d6c-0909ee11023a-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:13.842072 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:13.841988    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw" podUID="d571e7c1-d96c-4432-9665-032d0adb8371" containerName="main" containerID="cri-o://5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5" gracePeriod=45
Jul 10 14:06:14.091165 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.091139    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:06:14.254598 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.254569    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lk6p\" (UniqueName: \"kubernetes.io/projected/d571e7c1-d96c-4432-9665-032d0adb8371-kube-api-access-4lk6p\") pod \"d571e7c1-d96c-4432-9665-032d0adb8371\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") "
Jul 10 14:06:14.254754 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.254616    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-tmp-dir\") pod \"d571e7c1-d96c-4432-9665-032d0adb8371\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") "
Jul 10 14:06:14.254754 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.254677    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-home\") pod \"d571e7c1-d96c-4432-9665-032d0adb8371\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") "
Jul 10 14:06:14.254754 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.254710    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-dshm\") pod \"d571e7c1-d96c-4432-9665-032d0adb8371\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") "
Jul 10 14:06:14.254754 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.254739    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-model-cache\") pod \"d571e7c1-d96c-4432-9665-032d0adb8371\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") "
Jul 10 14:06:14.254997 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.254763    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d571e7c1-d96c-4432-9665-032d0adb8371-tls-certs\") pod \"d571e7c1-d96c-4432-9665-032d0adb8371\" (UID: \"d571e7c1-d96c-4432-9665-032d0adb8371\") "
Jul 10 14:06:14.254997 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.254969    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d571e7c1-d96c-4432-9665-032d0adb8371" (UID: "d571e7c1-d96c-4432-9665-032d0adb8371"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:14.255101 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.255085    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:14.255187 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.255162    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-home" (OuterVolumeSpecName: "home") pod "d571e7c1-d96c-4432-9665-032d0adb8371" (UID: "d571e7c1-d96c-4432-9665-032d0adb8371"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:14.255244 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.255226    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-model-cache" (OuterVolumeSpecName: "model-cache") pod "d571e7c1-d96c-4432-9665-032d0adb8371" (UID: "d571e7c1-d96c-4432-9665-032d0adb8371"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:14.257455 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.257427    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-dshm" (OuterVolumeSpecName: "dshm") pod "d571e7c1-d96c-4432-9665-032d0adb8371" (UID: "d571e7c1-d96c-4432-9665-032d0adb8371"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:06:14.257561 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.257507    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d571e7c1-d96c-4432-9665-032d0adb8371-kube-api-access-4lk6p" (OuterVolumeSpecName: "kube-api-access-4lk6p") pod "d571e7c1-d96c-4432-9665-032d0adb8371" (UID: "d571e7c1-d96c-4432-9665-032d0adb8371"). InnerVolumeSpecName "kube-api-access-4lk6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:06:14.257608 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.257559    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d571e7c1-d96c-4432-9665-032d0adb8371-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d571e7c1-d96c-4432-9665-032d0adb8371" (UID: "d571e7c1-d96c-4432-9665-032d0adb8371"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:06:14.356153 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.356129    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:14.356246 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.356158    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:14.356246 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.356172    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d571e7c1-d96c-4432-9665-032d0adb8371-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:14.356246 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.356188    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lk6p\" (UniqueName: \"kubernetes.io/projected/d571e7c1-d96c-4432-9665-032d0adb8371-kube-api-access-4lk6p\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:14.356246 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.356201    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d571e7c1-d96c-4432-9665-032d0adb8371-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:06:14.466392 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.466364    2554 generic.go:356] "Generic (PLEG): container finished" podID="d571e7c1-d96c-4432-9665-032d0adb8371" containerID="5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5" exitCode=0
Jul 10 14:06:14.466549 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.466412    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw" event={"ID":"d571e7c1-d96c-4432-9665-032d0adb8371","Type":"ContainerDied","Data":"5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5"}
Jul 10 14:06:14.466549 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.466442    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw" event={"ID":"d571e7c1-d96c-4432-9665-032d0adb8371","Type":"ContainerDied","Data":"0637ffde2bd18c476d031ffe7693e6c1630e0920270b1e0d105f0bca51f52245"}
Jul 10 14:06:14.466549 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.466459    2554 scope.go:117] "RemoveContainer" containerID="5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5"
Jul 10 14:06:14.466549 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.466415    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"
Jul 10 14:06:14.476447 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.476430    2554 scope.go:117] "RemoveContainer" containerID="5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5"
Jul 10 14:06:14.476710 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:06:14.476687    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5\": container with ID starting with 5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5 not found: ID does not exist" containerID="5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5"
Jul 10 14:06:14.476797 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.476717    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5"} err="failed to get container status \"5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5\": rpc error: code = NotFound desc = could not find container \"5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5\": container with ID starting with 5bfac56da35e8c18e825cc2a1026d16bfc01b555df4b1eaba12b65f97a4056e5 not found: ID does not exist"
Jul 10 14:06:14.500442 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.500418    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"]
Jul 10 14:06:14.507858 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.507794    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/sub-test-model-c21824-kserve-8596c84c95-q59xw"]
Jul 10 14:06:14.664528 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.664504    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bdd2f68-af6b-4db6-9d6c-0909ee11023a" path="/var/lib/kubelet/pods/8bdd2f68-af6b-4db6-9d6c-0909ee11023a/volumes"
Jul 10 14:06:14.664945 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:14.664930    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d571e7c1-d96c-4432-9665-032d0adb8371" path="/var/lib/kubelet/pods/d571e7c1-d96c-4432-9665-032d0adb8371/volumes"
Jul 10 14:06:28.422230 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.422192    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"]
Jul 10 14:06:28.422604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.422536    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bdd2f68-af6b-4db6-9d6c-0909ee11023a" containerName="main"
Jul 10 14:06:28.422604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.422547    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bdd2f68-af6b-4db6-9d6c-0909ee11023a" containerName="main"
Jul 10 14:06:28.422604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.422568    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d571e7c1-d96c-4432-9665-032d0adb8371" containerName="main"
Jul 10 14:06:28.422604 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.422574    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d571e7c1-d96c-4432-9665-032d0adb8371" containerName="main"
Jul 10 14:06:28.422733 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.422638    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d571e7c1-d96c-4432-9665-032d0adb8371" containerName="main"
Jul 10 14:06:28.422733 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.422648    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bdd2f68-af6b-4db6-9d6c-0909ee11023a" containerName="main"
Jul 10 14:06:28.425584 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.425565    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.429349 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.429326    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"kube-root-ca.crt\""
Jul 10 14:06:28.429484 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.429333    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"openshift-service-ca.crt\""
Jul 10 14:06:28.429484 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.429394    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"rate-test-model-ffeadb-kserve-self-signed-certs\""
Jul 10 14:06:28.429484 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.429438    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ffeadb\"/\"default-dockercfg-b78sx\""
Jul 10 14:06:28.436408 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.436383    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"]
Jul 10 14:06:28.467317 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.467291    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-dshm\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.467455 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.467347    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5nf7\" (UniqueName: \"kubernetes.io/projected/3bb164a3-f17a-4087-a1ff-b932b143369d-kube-api-access-c5nf7\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.467455 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.467401    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-model-cache\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.467559 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.467501    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-home\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.467559 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.467527    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bb164a3-f17a-4087-a1ff-b932b143369d-tls-certs\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.467559 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.467551    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-tmp-dir\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568501 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568476    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-home\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568506    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bb164a3-f17a-4087-a1ff-b932b143369d-tls-certs\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568525    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-tmp-dir\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568549    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-dshm\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568581    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5nf7\" (UniqueName: \"kubernetes.io/projected/3bb164a3-f17a-4087-a1ff-b932b143369d-kube-api-access-c5nf7\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568599    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-model-cache\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568929 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568911    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-model-cache\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568990 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568912    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-tmp-dir\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.568990 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.568980    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-home\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.570734 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.570714    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-dshm\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.571024 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.571006    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bb164a3-f17a-4087-a1ff-b932b143369d-tls-certs\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.581114 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.581095    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5nf7\" (UniqueName: \"kubernetes.io/projected/3bb164a3-f17a-4087-a1ff-b932b143369d-kube-api-access-c5nf7\") pod \"rate-test-model-ffeadb-kserve-799cb579bc-f8dbc\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") " pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.735590 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.735562    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:28.871999 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:28.871968    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"]
Jul 10 14:06:28.873960 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:06:28.873935    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3bb164a3_f17a_4087_a1ff_b932b143369d.slice/crio-ed77f614abd846e1a268ddf1c32ea84ec29927c0a846bfcedada51cf1775faa9 WatchSource:0}: Error finding container ed77f614abd846e1a268ddf1c32ea84ec29927c0a846bfcedada51cf1775faa9: Status 404 returned error can't find the container with id ed77f614abd846e1a268ddf1c32ea84ec29927c0a846bfcedada51cf1775faa9
Jul 10 14:06:29.536784 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:29.536742    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc" event={"ID":"3bb164a3-f17a-4087-a1ff-b932b143369d","Type":"ContainerStarted","Data":"8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af"}
Jul 10 14:06:29.536784 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:29.536789    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc" event={"ID":"3bb164a3-f17a-4087-a1ff-b932b143369d","Type":"ContainerStarted","Data":"ed77f614abd846e1a268ddf1c32ea84ec29927c0a846bfcedada51cf1775faa9"}
Jul 10 14:06:29.556925 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:29.556879    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc" podStartSLOduration=1.556862864 podStartE2EDuration="1.556862864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:06:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:06:29.555752612 +0000 UTC m=+2413.421879268" watchObservedRunningTime="2026-07-10 14:06:29.556862864 +0000 UTC m=+2413.422989537"
Jul 10 14:06:38.736017 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:38.735940    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:38.736017 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:38.735988    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:38.748559 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:38.748539    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:39.597638 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:39.597610    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:06:44.012123 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.012085    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"]
Jul 10 14:06:44.015717 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.015697    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.023291 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.023273    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"default-dockercfg-gjvr7\""
Jul 10 14:06:44.024613 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.024594    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"openshift-service-ca.crt\""
Jul 10 14:06:44.024721 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.024641    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"kube-root-ca.crt\""
Jul 10 14:06:44.024721 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.024693    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c21824\"/\"rate-test-model-c21824-kserve-self-signed-certs\""
Jul 10 14:06:44.030748 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.030717    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"]
Jul 10 14:06:44.098386 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.098357    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-model-cache\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.098386 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.098387    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2phpc\" (UniqueName: \"kubernetes.io/projected/3736bf08-d512-4c68-9133-0f4fcaf1034e-kube-api-access-2phpc\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.098557 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.098410    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-dshm\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.098557 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.098532    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-tmp-dir\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.098626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.098570    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-home\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.098626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.098608    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3736bf08-d512-4c68-9133-0f4fcaf1034e-tls-certs\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.199569 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.199537    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-model-cache\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.199569 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.199569    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2phpc\" (UniqueName: \"kubernetes.io/projected/3736bf08-d512-4c68-9133-0f4fcaf1034e-kube-api-access-2phpc\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.199741 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.199592    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-dshm\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.199741 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.199644    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-tmp-dir\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.199741 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.199671    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-home\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.199741 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.199701    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3736bf08-d512-4c68-9133-0f4fcaf1034e-tls-certs\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.199994 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.199957    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-model-cache\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.200051 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.200021    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-home\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.200090 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.200065    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-tmp-dir\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.201933 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.201912    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-dshm\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.202098 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.202082    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3736bf08-d512-4c68-9133-0f4fcaf1034e-tls-certs\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.209161 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.209142    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2phpc\" (UniqueName: \"kubernetes.io/projected/3736bf08-d512-4c68-9133-0f4fcaf1034e-kube-api-access-2phpc\") pod \"rate-test-model-c21824-kserve-966fd67bc-l8zlz\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") " pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.325203 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.325140    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:44.458250 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.458225    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"]
Jul 10 14:06:44.460477 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:06:44.460446    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3736bf08_d512_4c68_9133_0f4fcaf1034e.slice/crio-35e763580a99f8cec8d82f7537f0824a52a0db5b6eeaf851b179360639f262bf WatchSource:0}: Error finding container 35e763580a99f8cec8d82f7537f0824a52a0db5b6eeaf851b179360639f262bf: Status 404 returned error can't find the container with id 35e763580a99f8cec8d82f7537f0824a52a0db5b6eeaf851b179360639f262bf
Jul 10 14:06:44.612121 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.612044    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz" event={"ID":"3736bf08-d512-4c68-9133-0f4fcaf1034e","Type":"ContainerStarted","Data":"5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f"}
Jul 10 14:06:44.612121 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.612084    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz" event={"ID":"3736bf08-d512-4c68-9133-0f4fcaf1034e","Type":"ContainerStarted","Data":"35e763580a99f8cec8d82f7537f0824a52a0db5b6eeaf851b179360639f262bf"}
Jul 10 14:06:44.633179 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:44.633131    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz" podStartSLOduration=1.633117578 podStartE2EDuration="1.633117578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:06:44.631116355 +0000 UTC m=+2428.497243009" watchObservedRunningTime="2026-07-10 14:06:44.633117578 +0000 UTC m=+2428.499244231"
Jul 10 14:06:54.325593 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:54.325554    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:54.326016 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:54.325606    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:54.337628 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:54.337598    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:06:54.686420 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:06:54.686391    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:07:33.326138 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:33.326100    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"]
Jul 10 14:07:34.202158 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:34.202127    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"]
Jul 10 14:07:35.217579 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.217544    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7449-139996456287280"]
Jul 10 14:07:35.221282 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.221259    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996456287280"
Jul 10 14:07:35.223672 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.223655    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-cd94t\""
Jul 10 14:07:35.228273 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.228250    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7449-139996456287280"]
Jul 10 14:07:35.303750 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.303729    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8kvv\" (UniqueName: \"kubernetes.io/projected/ecf96028-8188-455f-ba21-4d9e643c1738-kube-api-access-d8kvv\") pod \"test-curl-7449-139996456287280\" (UID: \"ecf96028-8188-455f-ba21-4d9e643c1738\") " pod="openshift-ingress/test-curl-7449-139996456287280"
Jul 10 14:07:35.404715 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.404682    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8kvv\" (UniqueName: \"kubernetes.io/projected/ecf96028-8188-455f-ba21-4d9e643c1738-kube-api-access-d8kvv\") pod \"test-curl-7449-139996456287280\" (UID: \"ecf96028-8188-455f-ba21-4d9e643c1738\") " pod="openshift-ingress/test-curl-7449-139996456287280"
Jul 10 14:07:35.414465 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.414442    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8kvv\" (UniqueName: \"kubernetes.io/projected/ecf96028-8188-455f-ba21-4d9e643c1738-kube-api-access-d8kvv\") pod \"test-curl-7449-139996456287280\" (UID: \"ecf96028-8188-455f-ba21-4d9e643c1738\") " pod="openshift-ingress/test-curl-7449-139996456287280"
Jul 10 14:07:35.531348 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.531284    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996456287280"
Jul 10 14:07:35.666000 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.665966    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7449-139996456287280"]
Jul 10 14:07:35.668650 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:07:35.668620    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecf96028_8188_455f_ba21_4d9e643c1738.slice/crio-84be14ba9e73c8225682059e0fb295648a2f8222cfa64196e6ca127315c12eae WatchSource:0}: Error finding container 84be14ba9e73c8225682059e0fb295648a2f8222cfa64196e6ca127315c12eae: Status 404 returned error can't find the container with id 84be14ba9e73c8225682059e0fb295648a2f8222cfa64196e6ca127315c12eae
Jul 10 14:07:35.670366 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.670346    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 14:07:35.867904 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:35.867811    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7449-139996456287280" event={"ID":"ecf96028-8188-455f-ba21-4d9e643c1738","Type":"ContainerStarted","Data":"84be14ba9e73c8225682059e0fb295648a2f8222cfa64196e6ca127315c12eae"}
Jul 10 14:07:36.874223 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:36.874190    2554 generic.go:356] "Generic (PLEG): container finished" podID="ecf96028-8188-455f-ba21-4d9e643c1738" containerID="1cf6ddaffb5dad399e6c0a9f9368534fc42b76fe4fa63a229c79f55d37dd82b3" exitCode=0
Jul 10 14:07:36.874657 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:36.874229    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7449-139996456287280" event={"ID":"ecf96028-8188-455f-ba21-4d9e643c1738","Type":"ContainerDied","Data":"1cf6ddaffb5dad399e6c0a9f9368534fc42b76fe4fa63a229c79f55d37dd82b3"}
Jul 10 14:07:38.027401 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.027377    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996456287280"
Jul 10 14:07:38.068257 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.068226    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7449-139996456287280_ecf96028-8188-455f-ba21-4d9e643c1738/test-curl-7449-139996456287280/0.log"
Jul 10 14:07:38.094010 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.093980    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7449-139996456287280"]
Jul 10 14:07:38.096740 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.096717    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7449-139996456287280"]
Jul 10 14:07:38.127020 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.126999    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8kvv\" (UniqueName: \"kubernetes.io/projected/ecf96028-8188-455f-ba21-4d9e643c1738-kube-api-access-d8kvv\") pod \"ecf96028-8188-455f-ba21-4d9e643c1738\" (UID: \"ecf96028-8188-455f-ba21-4d9e643c1738\") "
Jul 10 14:07:38.129089 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.129058    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf96028-8188-455f-ba21-4d9e643c1738-kube-api-access-d8kvv" (OuterVolumeSpecName: "kube-api-access-d8kvv") pod "ecf96028-8188-455f-ba21-4d9e643c1738" (UID: "ecf96028-8188-455f-ba21-4d9e643c1738"). InnerVolumeSpecName "kube-api-access-d8kvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:07:38.228536 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.228502    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8kvv\" (UniqueName: \"kubernetes.io/projected/ecf96028-8188-455f-ba21-4d9e643c1738-kube-api-access-d8kvv\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:38.664192 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.664158    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf96028-8188-455f-ba21-4d9e643c1738" path="/var/lib/kubelet/pods/ecf96028-8188-455f-ba21-4d9e643c1738/volumes"
Jul 10 14:07:38.885049 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.885020    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996456287280"
Jul 10 14:07:38.885229 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:38.885066    2554 scope.go:117] "RemoveContainer" containerID="1cf6ddaffb5dad399e6c0a9f9368534fc42b76fe4fa63a229c79f55d37dd82b3"
Jul 10 14:07:48.345465 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.345427    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc" podUID="3bb164a3-f17a-4087-a1ff-b932b143369d" containerName="main" containerID="cri-o://8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af" gracePeriod=45
Jul 10 14:07:48.602458 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.602405    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:07:48.714247 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714219    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-model-cache\") pod \"3bb164a3-f17a-4087-a1ff-b932b143369d\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") "
Jul 10 14:07:48.714391 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714252    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-home\") pod \"3bb164a3-f17a-4087-a1ff-b932b143369d\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") "
Jul 10 14:07:48.714391 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714278    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-tmp-dir\") pod \"3bb164a3-f17a-4087-a1ff-b932b143369d\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") "
Jul 10 14:07:48.714391 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714362    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5nf7\" (UniqueName: \"kubernetes.io/projected/3bb164a3-f17a-4087-a1ff-b932b143369d-kube-api-access-c5nf7\") pod \"3bb164a3-f17a-4087-a1ff-b932b143369d\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") "
Jul 10 14:07:48.714542 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714407    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bb164a3-f17a-4087-a1ff-b932b143369d-tls-certs\") pod \"3bb164a3-f17a-4087-a1ff-b932b143369d\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") "
Jul 10 14:07:48.714542 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714468    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-dshm\") pod \"3bb164a3-f17a-4087-a1ff-b932b143369d\" (UID: \"3bb164a3-f17a-4087-a1ff-b932b143369d\") "
Jul 10 14:07:48.714542 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714510    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-model-cache" (OuterVolumeSpecName: "model-cache") pod "3bb164a3-f17a-4087-a1ff-b932b143369d" (UID: "3bb164a3-f17a-4087-a1ff-b932b143369d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:48.714698 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714541    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-home" (OuterVolumeSpecName: "home") pod "3bb164a3-f17a-4087-a1ff-b932b143369d" (UID: "3bb164a3-f17a-4087-a1ff-b932b143369d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:48.714698 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714552    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3bb164a3-f17a-4087-a1ff-b932b143369d" (UID: "3bb164a3-f17a-4087-a1ff-b932b143369d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:48.714795 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714725    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:48.714795 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714738    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:48.714795 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.714747    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:48.716491 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.716461    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-dshm" (OuterVolumeSpecName: "dshm") pod "3bb164a3-f17a-4087-a1ff-b932b143369d" (UID: "3bb164a3-f17a-4087-a1ff-b932b143369d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:48.716953 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.716930    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bb164a3-f17a-4087-a1ff-b932b143369d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3bb164a3-f17a-4087-a1ff-b932b143369d" (UID: "3bb164a3-f17a-4087-a1ff-b932b143369d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:07:48.717023 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.716947    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bb164a3-f17a-4087-a1ff-b932b143369d-kube-api-access-c5nf7" (OuterVolumeSpecName: "kube-api-access-c5nf7") pod "3bb164a3-f17a-4087-a1ff-b932b143369d" (UID: "3bb164a3-f17a-4087-a1ff-b932b143369d"). InnerVolumeSpecName "kube-api-access-c5nf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:07:48.815137 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.815111    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bb164a3-f17a-4087-a1ff-b932b143369d-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:48.815137 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.815136    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bb164a3-f17a-4087-a1ff-b932b143369d-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:48.815294 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.815147    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5nf7\" (UniqueName: \"kubernetes.io/projected/3bb164a3-f17a-4087-a1ff-b932b143369d-kube-api-access-c5nf7\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:48.934979 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.934952    2554 generic.go:356] "Generic (PLEG): container finished" podID="3bb164a3-f17a-4087-a1ff-b932b143369d" containerID="8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af" exitCode=0
Jul 10 14:07:48.935137 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.935006    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"
Jul 10 14:07:48.935137 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.935046    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc" event={"ID":"3bb164a3-f17a-4087-a1ff-b932b143369d","Type":"ContainerDied","Data":"8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af"}
Jul 10 14:07:48.935137 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.935086    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc" event={"ID":"3bb164a3-f17a-4087-a1ff-b932b143369d","Type":"ContainerDied","Data":"ed77f614abd846e1a268ddf1c32ea84ec29927c0a846bfcedada51cf1775faa9"}
Jul 10 14:07:48.935137 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.935104    2554 scope.go:117] "RemoveContainer" containerID="8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af"
Jul 10 14:07:48.944117 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.944097    2554 scope.go:117] "RemoveContainer" containerID="8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af"
Jul 10 14:07:48.944348 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:07:48.944330    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af\": container with ID starting with 8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af not found: ID does not exist" containerID="8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af"
Jul 10 14:07:48.944389 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.944357    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af"} err="failed to get container status \"8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af\": rpc error: code = NotFound desc = could not find container \"8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af\": container with ID starting with 8e05bf977bfc0c81a9fbc3386f6f6a9eaecbf84633bbafc2564cd0979adf11af not found: ID does not exist"
Jul 10 14:07:48.960196 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.960173    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"]
Jul 10 14:07:48.965367 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:48.965347    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ffeadb/rate-test-model-ffeadb-kserve-799cb579bc-f8dbc"]
Jul 10 14:07:49.221798 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.221709    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz" podUID="3736bf08-d512-4c68-9133-0f4fcaf1034e" containerName="main" containerID="cri-o://5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f" gracePeriod=45
Jul 10 14:07:49.477319 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.477268    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:07:49.521319 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521296    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-tmp-dir\") pod \"3736bf08-d512-4c68-9133-0f4fcaf1034e\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") "
Jul 10 14:07:49.521428 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521363    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-home\") pod \"3736bf08-d512-4c68-9133-0f4fcaf1034e\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") "
Jul 10 14:07:49.521428 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521400    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-model-cache\") pod \"3736bf08-d512-4c68-9133-0f4fcaf1034e\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") "
Jul 10 14:07:49.521543 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521427    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2phpc\" (UniqueName: \"kubernetes.io/projected/3736bf08-d512-4c68-9133-0f4fcaf1034e-kube-api-access-2phpc\") pod \"3736bf08-d512-4c68-9133-0f4fcaf1034e\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") "
Jul 10 14:07:49.521543 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521473    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3736bf08-d512-4c68-9133-0f4fcaf1034e-tls-certs\") pod \"3736bf08-d512-4c68-9133-0f4fcaf1034e\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") "
Jul 10 14:07:49.521543 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521523    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-dshm\") pod \"3736bf08-d512-4c68-9133-0f4fcaf1034e\" (UID: \"3736bf08-d512-4c68-9133-0f4fcaf1034e\") "
Jul 10 14:07:49.521543 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521533    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3736bf08-d512-4c68-9133-0f4fcaf1034e" (UID: "3736bf08-d512-4c68-9133-0f4fcaf1034e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:49.521740 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521621    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-home" (OuterVolumeSpecName: "home") pod "3736bf08-d512-4c68-9133-0f4fcaf1034e" (UID: "3736bf08-d512-4c68-9133-0f4fcaf1034e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:49.521740 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521671    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-model-cache" (OuterVolumeSpecName: "model-cache") pod "3736bf08-d512-4c68-9133-0f4fcaf1034e" (UID: "3736bf08-d512-4c68-9133-0f4fcaf1034e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:49.521836 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521791    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-model-cache\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:49.521836 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521810    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-tmp-dir\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:49.521947 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.521841    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-home\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:49.523513 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.523485    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3736bf08-d512-4c68-9133-0f4fcaf1034e-kube-api-access-2phpc" (OuterVolumeSpecName: "kube-api-access-2phpc") pod "3736bf08-d512-4c68-9133-0f4fcaf1034e" (UID: "3736bf08-d512-4c68-9133-0f4fcaf1034e"). InnerVolumeSpecName "kube-api-access-2phpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:07:49.523618 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.523534    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3736bf08-d512-4c68-9133-0f4fcaf1034e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3736bf08-d512-4c68-9133-0f4fcaf1034e" (UID: "3736bf08-d512-4c68-9133-0f4fcaf1034e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:07:49.523618 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.523562    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-dshm" (OuterVolumeSpecName: "dshm") pod "3736bf08-d512-4c68-9133-0f4fcaf1034e" (UID: "3736bf08-d512-4c68-9133-0f4fcaf1034e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 14:07:49.622402 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.622379    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2phpc\" (UniqueName: \"kubernetes.io/projected/3736bf08-d512-4c68-9133-0f4fcaf1034e-kube-api-access-2phpc\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:49.622402 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.622400    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3736bf08-d512-4c68-9133-0f4fcaf1034e-tls-certs\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:49.622536 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.622411    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3736bf08-d512-4c68-9133-0f4fcaf1034e-dshm\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:49.940976 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.940945    2554 generic.go:356] "Generic (PLEG): container finished" podID="3736bf08-d512-4c68-9133-0f4fcaf1034e" containerID="5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f" exitCode=0
Jul 10 14:07:49.941142 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.940993    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"
Jul 10 14:07:49.941142 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.941028    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz" event={"ID":"3736bf08-d512-4c68-9133-0f4fcaf1034e","Type":"ContainerDied","Data":"5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f"}
Jul 10 14:07:49.941142 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.941067    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz" event={"ID":"3736bf08-d512-4c68-9133-0f4fcaf1034e","Type":"ContainerDied","Data":"35e763580a99f8cec8d82f7537f0824a52a0db5b6eeaf851b179360639f262bf"}
Jul 10 14:07:49.941142 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.941086    2554 scope.go:117] "RemoveContainer" containerID="5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f"
Jul 10 14:07:49.951978 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.951936    2554 scope.go:117] "RemoveContainer" containerID="5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f"
Jul 10 14:07:49.952429 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:07:49.952403    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f\": container with ID starting with 5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f not found: ID does not exist" containerID="5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f"
Jul 10 14:07:49.952514 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.952438    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f"} err="failed to get container status \"5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f\": rpc error: code = NotFound desc = could not find container \"5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f\": container with ID starting with 5daeb41fbfd434d674275e1b8dc515f692309039eccf588a82dfddc6e2a3053f not found: ID does not exist"
Jul 10 14:07:49.967789 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.967767    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"]
Jul 10 14:07:49.974103 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:49.974084    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c21824/rate-test-model-c21824-kserve-966fd67bc-l8zlz"]
Jul 10 14:07:50.399010 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.398924    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7449-139996452367504"]
Jul 10 14:07:50.399360 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399342    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf96028-8188-455f-ba21-4d9e643c1738" containerName="test-curl-7449-139996456287280"
Jul 10 14:07:50.399453 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399364    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf96028-8188-455f-ba21-4d9e643c1738" containerName="test-curl-7449-139996456287280"
Jul 10 14:07:50.399453 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399396    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bb164a3-f17a-4087-a1ff-b932b143369d" containerName="main"
Jul 10 14:07:50.399453 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399406    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bb164a3-f17a-4087-a1ff-b932b143369d" containerName="main"
Jul 10 14:07:50.399453 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399416    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3736bf08-d512-4c68-9133-0f4fcaf1034e" containerName="main"
Jul 10 14:07:50.399453 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399425    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3736bf08-d512-4c68-9133-0f4fcaf1034e" containerName="main"
Jul 10 14:07:50.399701 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399535    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf96028-8188-455f-ba21-4d9e643c1738" containerName="test-curl-7449-139996456287280"
Jul 10 14:07:50.399701 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399552    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3736bf08-d512-4c68-9133-0f4fcaf1034e" containerName="main"
Jul 10 14:07:50.399701 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.399564    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bb164a3-f17a-4087-a1ff-b932b143369d" containerName="main"
Jul 10 14:07:50.404144 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.404123    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:50.406726 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.406704    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-cd94t\""
Jul 10 14:07:50.406999 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.406981    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7449-139996452367504"]
Jul 10 14:07:50.528806 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.528779    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbq2x\" (UniqueName: \"kubernetes.io/projected/247c42ae-359e-4519-ac94-f4360b9dc008-kube-api-access-dbq2x\") pod \"test-curl-7449-139996452367504\" (UID: \"247c42ae-359e-4519-ac94-f4360b9dc008\") " pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:50.630173 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.630143    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbq2x\" (UniqueName: \"kubernetes.io/projected/247c42ae-359e-4519-ac94-f4360b9dc008-kube-api-access-dbq2x\") pod \"test-curl-7449-139996452367504\" (UID: \"247c42ae-359e-4519-ac94-f4360b9dc008\") " pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:50.640470 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.640444    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbq2x\" (UniqueName: \"kubernetes.io/projected/247c42ae-359e-4519-ac94-f4360b9dc008-kube-api-access-dbq2x\") pod \"test-curl-7449-139996452367504\" (UID: \"247c42ae-359e-4519-ac94-f4360b9dc008\") " pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:50.664272 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.664215    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3736bf08-d512-4c68-9133-0f4fcaf1034e" path="/var/lib/kubelet/pods/3736bf08-d512-4c68-9133-0f4fcaf1034e/volumes"
Jul 10 14:07:50.664581 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.664568    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bb164a3-f17a-4087-a1ff-b932b143369d" path="/var/lib/kubelet/pods/3bb164a3-f17a-4087-a1ff-b932b143369d/volumes"
Jul 10 14:07:50.713432 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.713411    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:50.844913 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.844888    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7449-139996452367504"]
Jul 10 14:07:50.847701 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:07:50.847669    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod247c42ae_359e_4519_ac94_f4360b9dc008.slice/crio-6aa760a2d36d6fece0101837fbde1d0edc42c02cdcda6792b73e0ba7d2686599 WatchSource:0}: Error finding container 6aa760a2d36d6fece0101837fbde1d0edc42c02cdcda6792b73e0ba7d2686599: Status 404 returned error can't find the container with id 6aa760a2d36d6fece0101837fbde1d0edc42c02cdcda6792b73e0ba7d2686599
Jul 10 14:07:50.948181 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:50.948146    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7449-139996452367504" event={"ID":"247c42ae-359e-4519-ac94-f4360b9dc008","Type":"ContainerStarted","Data":"6aa760a2d36d6fece0101837fbde1d0edc42c02cdcda6792b73e0ba7d2686599"}
Jul 10 14:07:51.954191 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:51.954163    2554 generic.go:356] "Generic (PLEG): container finished" podID="247c42ae-359e-4519-ac94-f4360b9dc008" containerID="b345e526c05a6db7023ea7e142f4c62c34d59c9a3ab0b7501ddb92772e8e1abe" exitCode=0
Jul 10 14:07:51.954510 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:51.954222    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7449-139996452367504" event={"ID":"247c42ae-359e-4519-ac94-f4360b9dc008","Type":"ContainerDied","Data":"b345e526c05a6db7023ea7e142f4c62c34d59c9a3ab0b7501ddb92772e8e1abe"}
Jul 10 14:07:53.095117 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.095091    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:53.119312 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.119291    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7449-139996452367504_247c42ae-359e-4519-ac94-f4360b9dc008/test-curl-7449-139996452367504/0.log"
Jul 10 14:07:53.147206 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.147182    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7449-139996452367504"]
Jul 10 14:07:53.148665 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.148646    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbq2x\" (UniqueName: \"kubernetes.io/projected/247c42ae-359e-4519-ac94-f4360b9dc008-kube-api-access-dbq2x\") pod \"247c42ae-359e-4519-ac94-f4360b9dc008\" (UID: \"247c42ae-359e-4519-ac94-f4360b9dc008\") "
Jul 10 14:07:53.150565 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.150536    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/247c42ae-359e-4519-ac94-f4360b9dc008-kube-api-access-dbq2x" (OuterVolumeSpecName: "kube-api-access-dbq2x") pod "247c42ae-359e-4519-ac94-f4360b9dc008" (UID: "247c42ae-359e-4519-ac94-f4360b9dc008"). InnerVolumeSpecName "kube-api-access-dbq2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:07:53.153047 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.153028    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7449-139996452367504"]
Jul 10 14:07:53.250157 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.250094    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbq2x\" (UniqueName: \"kubernetes.io/projected/247c42ae-359e-4519-ac94-f4360b9dc008-kube-api-access-dbq2x\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:53.964324 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.964295    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:53.964324 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:53.964317    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6aa760a2d36d6fece0101837fbde1d0edc42c02cdcda6792b73e0ba7d2686599"
Jul 10 14:07:53.974321 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:07:53.974288    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7449-139996452367504\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="247c42ae-359e-4519-ac94-f4360b9dc008" pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:54.218667 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.218593    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"]
Jul 10 14:07:54.219039 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.218848    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56" podUID="d393706e-ad8c-4888-ac2a-0c3879cfffcf" containerName="maas-api" containerID="cri-o://ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc" gracePeriod=30
Jul 10 14:07:54.221059 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:07:54.221020    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7449-139996452367504\" is forbidden: User \"system:node:ip-10-0-135-59.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-59.ec2.internal' and this object" podUID="247c42ae-359e-4519-ac94-f4360b9dc008" pod="openshift-ingress/test-curl-7449-139996452367504"
Jul 10 14:07:54.664690 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.664662    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="247c42ae-359e-4519-ac94-f4360b9dc008" path="/var/lib/kubelet/pods/247c42ae-359e-4519-ac94-f4360b9dc008/volumes"
Jul 10 14:07:54.677399 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.677373    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:07:54.762721 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.762644    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d393706e-ad8c-4888-ac2a-0c3879cfffcf-openshift-service-ca\") pod \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") "
Jul 10 14:07:54.762877 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.762755    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls\") pod \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") "
Jul 10 14:07:54.762877 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.762843    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txcvr\" (UniqueName: \"kubernetes.io/projected/d393706e-ad8c-4888-ac2a-0c3879cfffcf-kube-api-access-txcvr\") pod \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\" (UID: \"d393706e-ad8c-4888-ac2a-0c3879cfffcf\") "
Jul 10 14:07:54.763032 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.763007    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d393706e-ad8c-4888-ac2a-0c3879cfffcf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d393706e-ad8c-4888-ac2a-0c3879cfffcf" (UID: "d393706e-ad8c-4888-ac2a-0c3879cfffcf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:07:54.763166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.763141    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d393706e-ad8c-4888-ac2a-0c3879cfffcf-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:54.765166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.765066    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d393706e-ad8c-4888-ac2a-0c3879cfffcf" (UID: "d393706e-ad8c-4888-ac2a-0c3879cfffcf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:07:54.765166 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.765088    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d393706e-ad8c-4888-ac2a-0c3879cfffcf-kube-api-access-txcvr" (OuterVolumeSpecName: "kube-api-access-txcvr") pod "d393706e-ad8c-4888-ac2a-0c3879cfffcf" (UID: "d393706e-ad8c-4888-ac2a-0c3879cfffcf"). InnerVolumeSpecName "kube-api-access-txcvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:07:54.864503 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.864477    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d393706e-ad8c-4888-ac2a-0c3879cfffcf-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:54.864503 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.864499    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txcvr\" (UniqueName: \"kubernetes.io/projected/d393706e-ad8c-4888-ac2a-0c3879cfffcf-kube-api-access-txcvr\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:07:54.971397 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.971368    2554 generic.go:356] "Generic (PLEG): container finished" podID="d393706e-ad8c-4888-ac2a-0c3879cfffcf" containerID="ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc" exitCode=0
Jul 10 14:07:54.971533 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.971412    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"
Jul 10 14:07:54.971533 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.971451    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56" event={"ID":"d393706e-ad8c-4888-ac2a-0c3879cfffcf","Type":"ContainerDied","Data":"ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc"}
Jul 10 14:07:54.971533 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.971502    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56" event={"ID":"d393706e-ad8c-4888-ac2a-0c3879cfffcf","Type":"ContainerDied","Data":"418480d46348238788c97b20586dbbb73054250eb5aa876e6a8131adaa1df20c"}
Jul 10 14:07:54.971533 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.971524    2554 scope.go:117] "RemoveContainer" containerID="ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc"
Jul 10 14:07:54.981042 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.981028    2554 scope.go:117] "RemoveContainer" containerID="ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc"
Jul 10 14:07:54.981276 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:07:54.981259    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc\": container with ID starting with ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc not found: ID does not exist" containerID="ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc"
Jul 10 14:07:54.981338 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.981284    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc"} err="failed to get container status \"ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc\": rpc error: code = NotFound desc = could not find container \"ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc\": container with ID starting with ec77ae4ed236885502c1447c30e31d10b9721c38abb428ed80e93360cb41abcc not found: ID does not exist"
Jul 10 14:07:54.996450 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:54.996426    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"]
Jul 10 14:07:55.001628 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:55.001609    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ffeadb-746bcd4565-smg56"]
Jul 10 14:07:56.664158 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:07:56.664123    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d393706e-ad8c-4888-ac2a-0c3879cfffcf" path="/var/lib/kubelet/pods/d393706e-ad8c-4888-ac2a-0c3879cfffcf/volumes"
Jul 10 14:08:02.700229 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:02.700190    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"]
Jul 10 14:08:02.700648 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:02.700473    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds" podUID="491d9207-8f95-46ff-899a-459eb3ac6d84" containerName="maas-api" containerID="cri-o://c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41" gracePeriod=30
Jul 10 14:08:02.947905 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:02.947877    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:08:03.017023 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.016946    2554 generic.go:356] "Generic (PLEG): container finished" podID="491d9207-8f95-46ff-899a-459eb3ac6d84" containerID="c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41" exitCode=0
Jul 10 14:08:03.017023 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.016989    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"
Jul 10 14:08:03.017229 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.016990    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds" event={"ID":"491d9207-8f95-46ff-899a-459eb3ac6d84","Type":"ContainerDied","Data":"c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41"}
Jul 10 14:08:03.017229 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.017134    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds" event={"ID":"491d9207-8f95-46ff-899a-459eb3ac6d84","Type":"ContainerDied","Data":"83c39eaf85060a605460a148cb56fc6b3a6f3bd4f6f67e6081b7c52a2febf532"}
Jul 10 14:08:03.017229 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.017171    2554 scope.go:117] "RemoveContainer" containerID="c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41"
Jul 10 14:08:03.025938 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.025915    2554 scope.go:117] "RemoveContainer" containerID="c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41"
Jul 10 14:08:03.026185 ip-10-0-135-59 kubenswrapper[2554]: E0710 14:08:03.026167    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41\": container with ID starting with c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41 not found: ID does not exist" containerID="c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41"
Jul 10 14:08:03.026243 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.026200    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41"} err="failed to get container status \"c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41\": rpc error: code = NotFound desc = could not find container \"c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41\": container with ID starting with c8bd1afb7a6cc4655d00c88ae9f4a134b7cc19cdc952bdebb0447652b5159e41 not found: ID does not exist"
Jul 10 14:08:03.034291 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.034272    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfbzj\" (UniqueName: \"kubernetes.io/projected/491d9207-8f95-46ff-899a-459eb3ac6d84-kube-api-access-dfbzj\") pod \"491d9207-8f95-46ff-899a-459eb3ac6d84\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") "
Jul 10 14:08:03.034367 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.034309    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491d9207-8f95-46ff-899a-459eb3ac6d84-openshift-service-ca\") pod \"491d9207-8f95-46ff-899a-459eb3ac6d84\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") "
Jul 10 14:08:03.034512 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.034492    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491d9207-8f95-46ff-899a-459eb3ac6d84-maas-api-tls\") pod \"491d9207-8f95-46ff-899a-459eb3ac6d84\" (UID: \"491d9207-8f95-46ff-899a-459eb3ac6d84\") "
Jul 10 14:08:03.034601 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.034582    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491d9207-8f95-46ff-899a-459eb3ac6d84-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "491d9207-8f95-46ff-899a-459eb3ac6d84" (UID: "491d9207-8f95-46ff-899a-459eb3ac6d84"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 14:08:03.034775 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.034753    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491d9207-8f95-46ff-899a-459eb3ac6d84-openshift-service-ca\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:08:03.036574 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.036466    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491d9207-8f95-46ff-899a-459eb3ac6d84-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "491d9207-8f95-46ff-899a-459eb3ac6d84" (UID: "491d9207-8f95-46ff-899a-459eb3ac6d84"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 14:08:03.036710 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.036684    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491d9207-8f95-46ff-899a-459eb3ac6d84-kube-api-access-dfbzj" (OuterVolumeSpecName: "kube-api-access-dfbzj") pod "491d9207-8f95-46ff-899a-459eb3ac6d84" (UID: "491d9207-8f95-46ff-899a-459eb3ac6d84"). InnerVolumeSpecName "kube-api-access-dfbzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 14:08:03.135333 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.135311    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfbzj\" (UniqueName: \"kubernetes.io/projected/491d9207-8f95-46ff-899a-459eb3ac6d84-kube-api-access-dfbzj\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:08:03.135333 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.135332    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491d9207-8f95-46ff-899a-459eb3ac6d84-maas-api-tls\") on node \"ip-10-0-135-59.ec2.internal\" DevicePath \"\""
Jul 10 14:08:03.340780 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.340748    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"]
Jul 10 14:08:03.343470 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:03.343438    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-c21824-598c7bd88b-hvbds"]
Jul 10 14:08:04.664059 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:04.664016    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="491d9207-8f95-46ff-899a-459eb3ac6d84" path="/var/lib/kubelet/pods/491d9207-8f95-46ff-899a-459eb3ac6d84/volumes"
Jul 10 14:08:10.203087 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:10.203016    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4d6fb599-c827b_1fd9d2fa-8ffe-41b5-83c2-f17d22df7008/authorino/0.log"
Jul 10 14:08:15.394659 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:15.394626    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7hrxb_01c24155-c090-46d2-8bb0-da82cfa18ff9/manager/0.log"
Jul 10 14:08:15.524800 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:15.524768    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7zgcc_f6f8f4cc-d87d-4017-960e-9399f98f6e4b/manager/0.log"
Jul 10 14:08:15.661561 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:15.661492    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5999cf9bb8-t6jnd_150a898a-affd-4421-aab9-76ad94e1efd1/manager/0.log"
Jul 10 14:08:15.787396 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:15.787371    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9cb55_32cbb613-77d9-4690-8c05-9f7a3f78ddfc/server/0.log"
Jul 10 14:08:15.924720 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:15.924696    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-24j8m_f6d92133-a0b4-4d37-ab3f-f2ef318d4664/manager/0.log"
Jul 10 14:08:16.332632 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:16.332564    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67ccc8fd-mf285_ac206005-8ec2-4863-b3a2-8f28b36f32a1/copy-manifests/0.log"
Jul 10 14:08:16.346544 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:16.346520    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67ccc8fd-mf285_ac206005-8ec2-4863-b3a2-8f28b36f32a1/manager/0.log"
Jul 10 14:08:16.726670 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:16.726643    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-84445c5967-8d52g_25af9b04-6c44-4828-a544-a4a06e5340c3/maas-api/0.log"
Jul 10 14:08:16.862451 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:16.862420    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-p44q4_f3d51f2c-0656-4c1c-a63a-0cbcafb335f6/postgres/0.log"
Jul 10 14:08:18.301692 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:18.301660    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4d6fb599-c827b_1fd9d2fa-8ffe-41b5-83c2-f17d22df7008/authorino/0.log"
Jul 10 14:08:18.431173 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:18.431140    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-gdl77_71f8461c-559b-449f-803d-362f7bc02571/manager/0.log"
Jul 10 14:08:18.560049 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:18.559971    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cvdt5_49eae6c4-a529-412e-93ca-899af3234504/manager/0.log"
Jul 10 14:08:18.806219 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:18.806192    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-896m6_7ff0a59a-e655-4860-8a76-e18108eac3c0/registry-server/0.log"
Jul 10 14:08:18.934456 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:18.934430    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fjvw9_f07c6756-fbde-41cd-836d-7d54d13f62a9/manager/0.log"
Jul 10 14:08:19.190012 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:19.189937    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5srn4_23ade6a6-8418-44e0-9ee3-f133b75cbeb7/manager/0.log"
Jul 10 14:08:19.828392 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:19.828361    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4978d7b8-6jwst_690dfb48-6746-450d-8684-9fde68fc75ab/kube-auth-proxy/0.log"
Jul 10 14:08:20.207578 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:20.207550    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g7kzk_bf6e7f19-79ab-456b-9552-95545fe4a133/payload-pre-processing/0.log"
Jul 10 14:08:20.838550 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:20.838519    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-j5r5q_6935803a-6131-4b13-aab7-8f2dd592ac55/main/0.log"
Jul 10 14:08:25.775353 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775321    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59nbp/must-gather-m5tfs"]
Jul 10 14:08:25.775746 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775667    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c42ae-359e-4519-ac94-f4360b9dc008" containerName="test-curl-7449-139996452367504"
Jul 10 14:08:25.775746 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775679    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c42ae-359e-4519-ac94-f4360b9dc008" containerName="test-curl-7449-139996452367504"
Jul 10 14:08:25.775746 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775689    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491d9207-8f95-46ff-899a-459eb3ac6d84" containerName="maas-api"
Jul 10 14:08:25.775746 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775695    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="491d9207-8f95-46ff-899a-459eb3ac6d84" containerName="maas-api"
Jul 10 14:08:25.775746 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775707    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d393706e-ad8c-4888-ac2a-0c3879cfffcf" containerName="maas-api"
Jul 10 14:08:25.775746 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775712    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d393706e-ad8c-4888-ac2a-0c3879cfffcf" containerName="maas-api"
Jul 10 14:08:25.776035 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775772    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="247c42ae-359e-4519-ac94-f4360b9dc008" containerName="test-curl-7449-139996452367504"
Jul 10 14:08:25.776035 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775783    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="491d9207-8f95-46ff-899a-459eb3ac6d84" containerName="maas-api"
Jul 10 14:08:25.776035 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.775790    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d393706e-ad8c-4888-ac2a-0c3879cfffcf" containerName="maas-api"
Jul 10 14:08:25.779041 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.779021    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:25.782031 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.782010    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59nbp\"/\"kube-root-ca.crt\""
Jul 10 14:08:25.782142 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.782118    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-59nbp\"/\"default-dockercfg-pvgbn\""
Jul 10 14:08:25.783304 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.783285    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59nbp\"/\"openshift-service-ca.crt\""
Jul 10 14:08:25.793943 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.793921    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59nbp/must-gather-m5tfs"]
Jul 10 14:08:25.909407 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.909368    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25895d2a-a7c0-48cc-bda2-977ec7918734-must-gather-output\") pod \"must-gather-m5tfs\" (UID: \"25895d2a-a7c0-48cc-bda2-977ec7918734\") " pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:25.909407 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:25.909404    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqpfq\" (UniqueName: \"kubernetes.io/projected/25895d2a-a7c0-48cc-bda2-977ec7918734-kube-api-access-pqpfq\") pod \"must-gather-m5tfs\" (UID: \"25895d2a-a7c0-48cc-bda2-977ec7918734\") " pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:26.010652 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:26.010619    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25895d2a-a7c0-48cc-bda2-977ec7918734-must-gather-output\") pod \"must-gather-m5tfs\" (UID: \"25895d2a-a7c0-48cc-bda2-977ec7918734\") " pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:26.010810 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:26.010671    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqpfq\" (UniqueName: \"kubernetes.io/projected/25895d2a-a7c0-48cc-bda2-977ec7918734-kube-api-access-pqpfq\") pod \"must-gather-m5tfs\" (UID: \"25895d2a-a7c0-48cc-bda2-977ec7918734\") " pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:26.010920 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:26.010901    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/25895d2a-a7c0-48cc-bda2-977ec7918734-must-gather-output\") pod \"must-gather-m5tfs\" (UID: \"25895d2a-a7c0-48cc-bda2-977ec7918734\") " pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:26.034414 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:26.034348    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqpfq\" (UniqueName: \"kubernetes.io/projected/25895d2a-a7c0-48cc-bda2-977ec7918734-kube-api-access-pqpfq\") pod \"must-gather-m5tfs\" (UID: \"25895d2a-a7c0-48cc-bda2-977ec7918734\") " pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:26.088291 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:26.088269    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59nbp/must-gather-m5tfs"
Jul 10 14:08:26.245238 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:26.245203    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59nbp/must-gather-m5tfs"]
Jul 10 14:08:26.249045 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:08:26.249006    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25895d2a_a7c0_48cc_bda2_977ec7918734.slice/crio-bf651f9dc57884244ec46e95d9536533ec77e76c00b19df1c73b4f4246a4b573 WatchSource:0}: Error finding container bf651f9dc57884244ec46e95d9536533ec77e76c00b19df1c73b4f4246a4b573: Status 404 returned error can't find the container with id bf651f9dc57884244ec46e95d9536533ec77e76c00b19df1c73b4f4246a4b573
Jul 10 14:08:27.130108 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:27.130074    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59nbp/must-gather-m5tfs" event={"ID":"25895d2a-a7c0-48cc-bda2-977ec7918734","Type":"ContainerStarted","Data":"bf651f9dc57884244ec46e95d9536533ec77e76c00b19df1c73b4f4246a4b573"}
Jul 10 14:08:28.138905 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:28.138860    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59nbp/must-gather-m5tfs" event={"ID":"25895d2a-a7c0-48cc-bda2-977ec7918734","Type":"ContainerStarted","Data":"7e376409a05c1c107ae86fd24bfb6158a94e695ab6353399f8a2a4df78b186db"}
Jul 10 14:08:28.138905 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:28.138917    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59nbp/must-gather-m5tfs" event={"ID":"25895d2a-a7c0-48cc-bda2-977ec7918734","Type":"ContainerStarted","Data":"30375471f96b90e90c7b43dd9157c8ef692fd1732ea03b47f6a470bcb48e54bb"}
Jul 10 14:08:28.756647 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:28.756619    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ggn9l_0acdcc26-44da-455e-a2df-a871c3c9a79f/global-pull-secret-syncer/0.log"
Jul 10 14:08:28.931629 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:28.931602    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l69wk_3a7c7869-c5b0-46ad-b31d-8dc7bfa9668e/konnectivity-agent/0.log"
Jul 10 14:08:29.033515 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:29.033431    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-59.ec2.internal_fcfe08aec5c2bd135c282f4562048962/haproxy/0.log"
Jul 10 14:08:34.309626 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:34.309590    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4d6fb599-c827b_1fd9d2fa-8ffe-41b5-83c2-f17d22df7008/authorino/0.log"
Jul 10 14:08:34.354725 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:34.354637    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-gdl77_71f8461c-559b-449f-803d-362f7bc02571/manager/0.log"
Jul 10 14:08:34.386522 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:34.386490    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cvdt5_49eae6c4-a529-412e-93ca-899af3234504/manager/0.log"
Jul 10 14:08:34.455886 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:34.455857    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-896m6_7ff0a59a-e655-4860-8a76-e18108eac3c0/registry-server/0.log"
Jul 10 14:08:34.618847 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:34.618736    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fjvw9_f07c6756-fbde-41cd-836d-7d54d13f62a9/manager/0.log"
Jul 10 14:08:34.734791 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:34.734765    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5srn4_23ade6a6-8418-44e0-9ee3-f133b75cbeb7/manager/0.log"
Jul 10 14:08:36.805718 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:36.805686    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4hgg_9447a7eb-4121-439a-95a8-6fbe7f162689/node-exporter/0.log"
Jul 10 14:08:36.828408 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:36.828368    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4hgg_9447a7eb-4121-439a-95a8-6fbe7f162689/kube-rbac-proxy/0.log"
Jul 10 14:08:36.868834 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:36.868789    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h4hgg_9447a7eb-4121-439a-95a8-6fbe7f162689/init-textfile/0.log"
Jul 10 14:08:38.591321 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.591243    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59nbp/must-gather-m5tfs" podStartSLOduration=12.795397102 podStartE2EDuration="13.591215671s" totalImagesPullingTime="795.818569ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:08:28.15807821 +0000 UTC m=+2532.024204867" watchObservedRunningTime="2026-07-10 14:08:38.591215671 +0000 UTC m=+2542.457342326"
Jul 10 14:08:38.593012 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.592980    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"]
Jul 10 14:08:38.599216 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.599192    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.608027 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.608002    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"]
Jul 10 14:08:38.630029 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.629995    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-sys\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.630242 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.630223    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-podres\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.630414 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.630397    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-proc\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.630578 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.630558    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg4tf\" (UniqueName: \"kubernetes.io/projected/7707dafb-6560-45ce-96eb-c501d08797b4-kube-api-access-sg4tf\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.630718 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.630702    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-lib-modules\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.731603 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731568    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-proc\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.731786 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731628    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg4tf\" (UniqueName: \"kubernetes.io/projected/7707dafb-6560-45ce-96eb-c501d08797b4-kube-api-access-sg4tf\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.731786 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731663    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-lib-modules\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.731786 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731683    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-proc\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.731786 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731701    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-sys\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.731786 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731727    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-podres\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.732095 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731815    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-sys\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.732095 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731869    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-lib-modules\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.732095 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.731874    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7707dafb-6560-45ce-96eb-c501d08797b4-podres\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.742629 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.742595    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg4tf\" (UniqueName: \"kubernetes.io/projected/7707dafb-6560-45ce-96eb-c501d08797b4-kube-api-access-sg4tf\") pod \"perf-node-gather-daemonset-phmlh\" (UID: \"7707dafb-6560-45ce-96eb-c501d08797b4\") " pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:38.807957 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.807925    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-99r2k_e24d0483-f492-46f5-a9e1-40b4638b8b5d/networking-console-plugin/0.log"
Jul 10 14:08:38.913466 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:38.913389    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:39.089533 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:39.088900    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"]
Jul 10 14:08:39.090592 ip-10-0-135-59 kubenswrapper[2554]: W0710 14:08:39.090562    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7707dafb_6560_45ce_96eb_c501d08797b4.slice/crio-2a5ffa8785a45e324cb4b5312f81595c48c3d532ff4c44945f7bb62063c43eae WatchSource:0}: Error finding container 2a5ffa8785a45e324cb4b5312f81595c48c3d532ff4c44945f7bb62063c43eae: Status 404 returned error can't find the container with id 2a5ffa8785a45e324cb4b5312f81595c48c3d532ff4c44945f7bb62063c43eae
Jul 10 14:08:39.230736 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:39.230698    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh" event={"ID":"7707dafb-6560-45ce-96eb-c501d08797b4","Type":"ContainerStarted","Data":"6d42188746b2f12fe02ee69f864a49fee53cb3d68a01816c422b2c8a55c7fbe7"}
Jul 10 14:08:39.230954 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:39.230762    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh" event={"ID":"7707dafb-6560-45ce-96eb-c501d08797b4","Type":"ContainerStarted","Data":"2a5ffa8785a45e324cb4b5312f81595c48c3d532ff4c44945f7bb62063c43eae"}
Jul 10 14:08:39.230954 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:39.230880    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:39.249463 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:39.249416    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh" podStartSLOduration=1.249403033 podStartE2EDuration="1.249403033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:08:39.248694951 +0000 UTC m=+2543.114821610" watchObservedRunningTime="2026-07-10 14:08:39.249403033 +0000 UTC m=+2543.115529687"
Jul 10 14:08:41.579883 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:41.579850    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-brqtm_f2a7807f-1566-4212-8e13-1e82c45eca50/dns/0.log"
Jul 10 14:08:41.604917 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:41.604888    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-brqtm_f2a7807f-1566-4212-8e13-1e82c45eca50/kube-rbac-proxy/0.log"
Jul 10 14:08:41.781931 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:41.781904    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g4sfq_fc717d08-6567-4c6b-9cdd-1d260730c235/dns-node-resolver/0.log"
Jul 10 14:08:42.397465 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:42.397434    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6d7b474fb8-bw7wm_59cc6f56-3d4f-4948-b857-5f18de51b88b/registry/0.log"
Jul 10 14:08:42.454158 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:42.454129    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kv96s_5dc9cd33-0fcc-4350-91b3-a9ce3df6b689/node-ca/0.log"
Jul 10 14:08:43.696914 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:43.696888    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4978d7b8-6jwst_690dfb48-6746-450d-8684-9fde68fc75ab/kube-auth-proxy/0.log"
Jul 10 14:08:43.857149 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:43.857107    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g7kzk_bf6e7f19-79ab-456b-9552-95545fe4a133/payload-pre-processing/0.log"
Jul 10 14:08:44.699934 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:44.699907    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wnk2p_025769c9-3932-4cd0-97cb-96e766b3157c/serve-healthcheck-canary/0.log"
Jul 10 14:08:45.245423 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:45.245391    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjb42_55a58a5f-0542-4a7c-9ebf-403fc515cd0d/kube-rbac-proxy/0.log"
Jul 10 14:08:45.246915 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:45.246885    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-59nbp/perf-node-gather-daemonset-phmlh"
Jul 10 14:08:45.277045 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:45.277019    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjb42_55a58a5f-0542-4a7c-9ebf-403fc515cd0d/exporter/0.log"
Jul 10 14:08:45.303406 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:45.303383    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjb42_55a58a5f-0542-4a7c-9ebf-403fc515cd0d/extractor/0.log"
Jul 10 14:08:48.025516 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:48.025486    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7hrxb_01c24155-c090-46d2-8bb0-da82cfa18ff9/manager/0.log"
Jul 10 14:08:48.197968 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:48.197934    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7zgcc_f6f8f4cc-d87d-4017-960e-9399f98f6e4b/manager/0.log"
Jul 10 14:08:48.320945 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:48.320869    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5999cf9bb8-t6jnd_150a898a-affd-4421-aab9-76ad94e1efd1/manager/0.log"
Jul 10 14:08:48.343349 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:48.343323    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9cb55_32cbb613-77d9-4690-8c05-9f7a3f78ddfc/server/0.log"
Jul 10 14:08:48.428087 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:48.428060    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-24j8m_f6d92133-a0b4-4d37-ab3f-f2ef318d4664/manager/0.log"
Jul 10 14:08:48.637776 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:48.637698    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67ccc8fd-mf285_ac206005-8ec2-4863-b3a2-8f28b36f32a1/manager/0.log"
Jul 10 14:08:48.659616 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:48.659588    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d67ccc8fd-mf285_ac206005-8ec2-4863-b3a2-8f28b36f32a1/copy-manifests/0.log"
Jul 10 14:08:49.066614 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:49.066586    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4978d7b8-6jwst_690dfb48-6746-450d-8684-9fde68fc75ab/kube-auth-proxy/0.log"
Jul 10 14:08:49.213996 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:49.213963    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g7kzk_bf6e7f19-79ab-456b-9552-95545fe4a133/payload-pre-processing/0.log"
Jul 10 14:08:50.593147 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:50.593114    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4978d7b8-6jwst_690dfb48-6746-450d-8684-9fde68fc75ab/kube-auth-proxy/0.log"
Jul 10 14:08:50.720062 ip-10-0-135-59 kubenswrapper[2554]: I0710 14:08:50.720034    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-g7kzk_bf6e7f19-79ab-456b-9552-95545fe4a133/payload-pre-processing/0.log"